API Guide

This document covers the use of the Secplugs REST API developers can use to access the scanning services programmatically.

You can use Secplugs services without writing code by using one of the many Plugin Types.

Also, many developers prefer to use the Language Kits which are a simpler way to get started.

See Plugin Gallery to see all the Plugins and Kits available.

Quick Start

curl -X GET "https://api.live.secplugs.com/security/web/score?url=https%3A%2F%2Fwww.example.com" -H "accept: application/json" -H "x-api-key: ILbW1sKwPs8CWO76E8ex47TR7zCZ2a8L50oq7sPI"

Swagger Documentation

The API is formally documented in swagger, ensuring you choose the ‘security’ section of the swagger tool to see the endpoints. The security API is the only api that plugins should interact with.

http://info.live.secplugs.com/

You may want to browse the swagger interface before reading this document. Before you can try the swagger interface you will need to have read the section on API keys as you will need an api key.

Sample Code

There is working sample code in a number of languages in the github repos below

https://github.com/search?q=secplugs-sample

The examples include a bash and curl example, a NodeJS example and Python script to show the basic usage of the api. These examples will run out of the box and are a great place to start.

Test Samples

There is a variety of test samples (emails, files and urls) on the Secplugs portal below

https://secplugs.com/index.php/test-sample-page

API Keys

All security api calls are authenticated with a standard header x-api-key header.

This api-key also refers to a configuration held in the brokerage that defines what analysis is done on the threat object when it is sent for scanning (e.g. what security vendors are used for the analysis). The plugin should be agnostic to this.

Anonymous Default API Key

Plugins should ship with a default ‘anonymous’ api key which can be embedded (it is not secret, it is public domain). This ensures the plugin will work out of the box against the public anonymous account.

For production each plugin type should have its own default api key, this is because each plugin type will have different default configuration setup.

For development you can use the anonymous generic api key below.

ILbW1sKwPs8CWO76E8ex47TR7zCZ2a8L50oq7sPI

Note: The above key is public domain and does not need to be kept secret. They are protected from abuse with usage quotas.

Mock API Keys & Test Points {#mock-api-keys-&-test-points}

Within the brokerage there is a mock vendor implementation which is explicitly created for testing.

The mock vendor implementation has the following behaviour:

  • File APIs will report a malware verdict ( score = 10 ) if EICAR is used
  • Email APIs will report a suspicious verdict ( score = 30 ) if a GTUBE is used
  • The Web APIs will return a malware verdict score ( score = 10 ) if the url contains example.com/test_malware
  • The Web API will return a suspicious verdict ( score = 30 ) if the url contains example.com/test_suspicous
  • Everything else will return clean (score = 70 )

There are two mock API Keys that are configured to use the mock vendor implementation as follows:

Synchronous Testing

The below api key will return the mock result synchronously - i.e. immediately after the GET security api call.

P7QnYz39Vo6BfHY6P8QvFapwptrto0CyaflSZHGW

Asynchronous Testing

The below key will return mock results asynchronous testing and will return a pending result until the 5th call when it will return a success result with the verdicts above.

GW5sb8sj8D9CtvVrjsTC22FNljxhoVuL1UoM6fFL

Note: The above keys are public domain and do not need to be kept secret. They are protected from abuse with usage quotas.

Customer API Keys

When a user signs up to the service they create api keys that are applied to the plugins to use in place of the default anonymous key. This allows the paying customer private and premium use of the services.

The plugin should direct the anonymous user to the sign up process. See the landing pages section for details.

The plugin should allow the user to provide the api key for use, store it securely and use it in the place of the default key. The plugin does not need to allow roll back to the default key.

In every other way the customer api key is the same as the default key.

API Endpoints

See swagger for the API endpoint specifics, this section gives an overview.

Overview

The general usage of the security APIs is that you call a scan or score endpoint with the appropriate API key and threat object specified.

The API will respond with a report_id which then needs to be polled against the report end point until the job is complete or a failure occurs.

Sometimes a threat object such as a file or email will need to be uploaded - see the uploading objects section.

Healthcheck

The healthcheck endpoint can be used to test connectivity and validate an api key is correct. It will return HTTP 200 and a json object showing the brokerage build and status.

   
GET/healthcheck Service status check

Score and Scan

The security api provides endpoints for different threat object types e.g. file, url and email.

Each has a score, a quickscan and a deepscan endpoint.

These are designed to be coarse levels of trade offs between depth of information vs. the latency and cost to generate it as follows:

  • score: Use when the only thing that is needed is a fast & efficient verdict on the threat object. High volume, mostly clean. Lowest cost. Audience is normally code, the vast majority of score responses are clean and not seen by a human. Fast and efficient.
  • quickscan - Use when more contextual information is needed. Medium volume, perhaps more suspicious. Medium cost. The results may be seen by a human - typically an end user (e.g. tom sees a detection name in a pop up) or an admin (a quarantine view) its ok to spend a bit longer / pay a bit more.
  • deepscan - Use to throw the book at it. Forensics. Audiences are always human, typically for the admin or SOC person, expert, admin. Highest cost. No expense spared and take as long as you need. Dig into it, tell me all …. about it.

As an example design pattern a plugin may call file/score, and then if it’s suspicious, do file/quickscan, then if its malware do a file/deepscan to present the report to the user.

You shouldn’t think of some of the endpoints being synchronous vs. asynchronous and some being block lists vs. scan requests.

All of the end points may return asynchronous results, but it’s more likely to get a synchronous result back from a score endpoint - this is because a verdict can often be generated very quickly, but not always.

All of the file and email endpoints may require an upload of the object but it is more likely that the score endpoint will not need it (for example if it uses a block list to return the verdict).

The scan and score endpoints can and should (but is not mandatory) also contain a scancontext parameter. See the Context Data section for more information.

The scan and score endpoints can also take a vendor_cfg parameter. Plugins should not use this parameter.

File Endpoints

The file analysis endpoints are below. They all take a SHA256 of the file. See the section on uploading objects to understand when and how files are uploaded.

   
GET​/file​/score Gets a threat score for a file
GET/file​/quickscan Get a score and a quick threat analysis report for a file
GET/file​/deepscan Get a score and a deep threat analysis report for a file

Web Endpoints

The web analysis endpoints are below. They all take a url which must include the scheme (e.g. https or http).

   
GET/web​/score Get a threat score for a url
GET/web​/quickscan Get a score and a quick threat analysis report for a url
GET​/web​/deepscan Get a score and a deep threat analysis report for a url

Email Endpoints

The email analysis endpoints are below. They all take an email_id. An email_id is a user defined unique id of the RFC822 email. It can be the sha256 of the email but it does not have to be. It is up to the caller to ensure uniqueness. Emails have the same object uploading paradigm as files. See the uploading objects section to understand how and when emails should be uploaded.

   
GET/email​/score Get a threat score for an email mime stream
GET/email​/quickscan Get a score and a quick threat analysis report for an email
GET/email​/deepscan Get a score and a deep threat analysis report for an email
GET/email​/attachmentscan Get a score and a quick threat analysis report for the attachments
GET/email​/bodyscan Get a score and a quick threat analysis report for the body

Uploading a Threat Object

The file and email apis may require the file or email for analysis. If the brokerage does not have a cached copy of the object and it is needed to complete the request a 404 not found will be issued. The plugin needs to upload the object and retry to get the results of the analysis.

A plugin that only wants to use the APIs as a lookup may choose not to upload the full object and proceed appropriately treating the 404 as an ‘unknown’.

Uploading uses the AWS pre-signed paradigm to upload data. This means the data is fielded by AWS s3 infrastructure rather than the AWS Gateway that the security API is exposed. Google AWS Presigned Uploads to read more about the design pattern.

To upload an object use the GET upload APIs to retrieve a pre-signed post to upload to. There is one for email and file.

   
GET/file​/upload Upload a file so it can be analysed
GET/email​/upload Upload a email so it can be analysed

See the sample code for working examples of how to do this.

Retrieving Results

The security APIs are all asynchronous in that they will return a report id that can be polled to receive the results. After making the GET call the plugin should then poll the report api until the status is successful to retrieve the results.

Below is the report endpoint

   
GET/report​/{report_id} Get the report for the specified report id

It may be that you also get a score and a repot_json in the initial response if the brokerage was able to complete the request synchronously (e.g. from cache) but your implementation should expect to handle the asynchronous case.

You only need to wait for the result if you are going to act on it (block, warn, prompt etc) - i.e. a plugin that does not act on the verdict does not need to wait to retrieve the full report instead the report id can be saved and formatted into a portal url for the user to see the report at a later time. If the user accesses that url before the report is ready the portal will handle the waiting experience.

A plugin that has very low latency requirements may decide to timeout on waits and fall into a follow up non blocking mode for threat objects that are not cached or don’t have a fast response.

Providing Context Data

Context data is passed up with a scan request to provide context. This may be useful for inline analysis and later data mining in the activity store.

Arbitrary name value pairs can be sent but the following our current standards

   
Key Value
plugin_version One to four dot-separated integers identifying the version of the plugin e.g. ‘3.1.2.4567’ or ‘2.8’
client_uuid A unique RFC4122 v4 uuid for the client generated at install time e.g. “ae1b7a04-4af3-4569-b132-8c1528cb34e8”
file_name The file name of the scan object (file or email) e.g. ‘notepad.exe’
subject The email subject if scanning an email e.g. ‘RE: urgent document’

You need to provide the context on the GET request of the security api call. Create a json object with the keys and values you want to pass, convert to a url encoded string and set is as the value for the scancontext url parameter. See the code samples to see how this is done.

Settings

A plugin can get settings via the settings API endpoint

   
GET/settings Get the settings for the plugin

The settings are specific to plugin type and are set by the user in the portal

API Return Values

This section provides a guide to the json return values from the endpoints.

Scan & Score {#scan-&-score}

When a score or can analysis endpoint is called it will return the following fields

   
Value Description
status The status of the request, either ‘success’, ‘pending’, ‘failure’
threat_object Object containing the threat object for analysis (e.g. { “url” : “https://www.a.com”}
scan_context Object containing the data supplied in scancontext param
datetime The epoch time the request started (e.g. 1608290649.0249567)
account_id The account that will be debited and that the api_key belongs to
api_key The ai key used for the request
meta_data Object with with additional data (vendors, credit, plugin info etc)
report_id The id of the report. This is a cryptographically strong ID

The most important values to the plugin are the status and the report id, the other values are informational and subject to change.

The report id is the unique id that the plugin should use to poll the report endpoint to check progress and get the results when completed.

If the status is ‘success’ then the call was satisfied synchronously the response will also contain the fields returned by the report endpoint. However the report endpoint can still be called to retrieve the data again.

If the status is ‘pending’ the report id should be used to poll the report endpoint for completion and results

If the status is ‘failure’ the analysis failed and there should be additional information on the error in the json response

Report

When polling the report endpoint the plugin will see the following top level values in the json response alongside the values already seen in the submission endpoints.

   
Value Description
status The status of the request, either ‘success’, ‘pending’, ‘failure’
score The score of the threat score of the object
verdict The verdict (based on the score)
duration The duration of the analysis
json_report Only for the ‘scan’ endpoints. This data is vendor specific and renderable by portal only.

Status is the flow control value, score and or verdict are actionable, duration is informational and json_report should not be parsed by the plugin. Other metadata fields will also be present but are not meant for plugin use.

If the status is ‘success’ then the above fields will be available and the plugin can proceed accordingly. See the Interpreting Threat Score and Verdict sections.

If status is pending the plugin should poll again.

If the status is ‘failure’ the plugin should invoke error handling.

API HTTP Codes

Swagger defines the error codes in more detail but the key HTTP return codes are here

   
Code Description
403 The api key was not valid or the endpoint does not exist
429 The api key has exceeded quota or run out of credit
200 Everything is ok.
404 File or Email not found. Typically it needs uploading.
4xx Some error on the request side
5xx Some error on the server side - look for json body for vendor error

Note: 404 in the above is not an error case and can be the normal part of a file or email analysis flow where the object has not been uploaded before.

Interpreting Threat Score & Verdict {#interpreting-threat-score-&-verdict}

When an analysis is complete all the apis will return a threat score and verdict.

The score is in a range from 0-100 and the verdict is a mapping of that score to sub ranges as per the table below.

     
Range Verdict Description
0-20 Malware Strong reason to believe it is malicious plugin should block and or alert.
21-40 Suspicious Reason to believe, it’s dangerous or unwanted (PUA/ SPAM). Plugin should block or warn.
41-60 Untrusted No reason to believe malice but the analysis tried and failed to establish trust
61-80 Clean No reason to believe malice. This is the default fail safe value.
81-100 Trusted The analysis was able to establish trust (certificate or trust list)

Some of the simplest vendor configurations will not try and establish trust and so results will only be in the malware or clean ranges. Others have more fine grained outputs and map to the full range. The fail policy is fail open and failures will map to the clean range.

Spam and configurations that distinguish files as PUA/PUP will land in the suspicious range.

Configurations that identify phish emails or urls will land in the malware range.

Displaying Vendor Specific Reports

The Score endpoints only return a score and verdict, these can be interpreted and actioned directly by the plugin.

But the scan endpoints also respond with vendor specific report data serialised in the json_report object. This data is vendor specific and subject to change, a plugin should not attempt to parse it. Instead redirect a user to the portal landing urls to display the reports. The portal has vendor specific PHP twig templates that are kept up to date and can render vendor specific details.

Plugin Values

A plugin has a number of important values it should create or maintain and use in interactions with the security api and portal landing pages

Client UUID

This is an rfc4122 compliant unique id that should be created at installation time.

It is used to identify a unique plugin installation for example to distinguish between plugins use the same api_key in telemetry

Example:

‘6491f6d0-40b5-11eb-b378-0242ac130002

Plugin Version

This is the version of the plugin created at build time.

Use same the format as https://developer.chrome.com/docs/extensions/mv2/manifest/version/

Example:

2.5.6 or 1.1.456.789

###

Portal Landing Pages

To integrate with the portal your plugin should redirect the user to the following urls where appropriate.

The URLs can take parameters as GET or POST with the later being more secure. The pages will then redirect to the appropriate material in the portal.

View Report - Render a report give an report_id
Path https://secplugs.com/plugin_landing/viewreport.php
report_id The report id to render
View Scan History - Show scan activity history of the plugin
Path https://secplugs.com/plugin_landing/viewscanhistory.php
client_uuid The client uuid generated by the plugin at installation time
api_key Current (default, anonymous) api_key in use. Portal will match plugin type.
Detection / Warning - Given a report_id show a detection / warning page
Path https://secplugs.com/plugin_landing/showalert.php
report_id The report id to render
Sign Up - Start the user flow to sign up and become a registered user
Path https://secplugs.com/plugin_landing/signup.php
client_uuid The client uuid generated by the plugin at installation time
api_key Current (default, anonymous) api_key in use. Portal will match plugin type.
Sign In - Direct the user to the sign in page
Path https://secplugs.com/plugin_landing/signin.php
Visit Us- Direct the user to the web site
Path https://secplugs.com/plugin_landing/visitus.php
Get API Key - Start the user flow to get an api key for this plugin
Path https://secplugs.com/plugin_landing/getapikey.php
client_uuid The client uuid generated by the plugin at installation time
api_key Current (default, anonymous) api_key in use. Portal will match plugin type.
Learn More - Take the user to pages about the plugin and benefits of signing up
Path https://secplugs.com/plugin_landing/learnmore.php
client_uuid The client uuid generated by the plugin at installation time
api_key Current (default, anonymous) api_key in use. Portal will match plugin type.