API Guide

This document is for you if you are a developer that wants to use the Secplugs REST API directly to access the scanning services programmatically.

You can use Secplugs services without writing code by using one of the many Plugins.

Also, many developers prefer to use the Language Kits which are a simpler way to get started.

See Plugin Gallery to see all the Plugins and Kits available.

Contents

Open API 3.0 Documentation

The API is formally documented in Open API 3.0.

Go to the API Reference page to see a simplified list of end points that you can try in the browser.

There is also a complete native Swagger UI on the link below. Ensure you choose the ‘security’ section of the swagger tool to see the relevant endpoints. The security API is the only api that is documented here and the only API that will work with the API Keys from this portal. http://info.live.secplugs.com/

Sample Code

There is working sample code in a number of languages in the GitHub repos below

https://github.com/search?q=secplugs-sample

The examples include a bash and curl example, a NodeJS example and Python script to show the basic usage of the api.

Quick Start Examples

Use Curl to get the score of a url:

curl -X GET "https://api.live.secplugs.com/security/web/score?url=https%3A%2F%2Fwww.example.com" -H "accept: application/json" -H "x-api-key: ILbW1sKwPs8CWO76E8ex47TR7zCZ2a8L50oq7sPI"

Use Curl to up load and scan a file:

curl -X 'POST' 'https://proxy.secplugs.com/file/jsonupload' -H 'accept: application/json' -H 'Content-Type: multipart/form-data' -F 'file=@file.bin;type=application/octet-stream'

About API Keys

All security API calls are authenticated with a standard header x-api-key header.

As well as authentication, API Keys are associated with a configuration that defines what analysis is done on the threat object when it is sent for scanning. For example you can change what scanners are used to scan.

You can use the default API keys that are freely available and have a default configuration or you can register and create your own API keys that you can control and can configure yourself. See How To Create A Key

Default API Key

Plugins come pre configured out of the box with a default ‘anonymous’ API key.

Some Plugins have a specific default API key this is pre configured with a custom configuration appropriate for them.

For development you can use the anonymous generic API key below.

ILbW1sKwPs8CWO76E8ex47TR7zCZ2a8L50oq7sPI

Note: The above API key is public domain and does not need to be kept secret. They are protected from abuse with usage quotas.

Performing a Scan

To see a list of end points and try them out see the API Reference page.

The general paradigm is that you call a scan or score endpoint with the appropriate API key and threat object specified.

The API will respond with a report_id which then needs to be polled against the report end point until the job is complete or a failure occurs.

Sometimes a threat object such as a file or email will need to be uploaded - see the uploading objects section.

Scan Helpers

The Scan Helper End Points end points are the simplest way to use the API. They are a wrapper for the standard File Scanning end points. Designed for use from simple scripting languages or no code tools with limited functionality. They take care of SHA256, upload to storage and polling for the scan result, so you can just upload the file and get the result.

However if you want more control on type of scan, how you manage file uploading and results retrieval or you want to scan Emails or URLs you’ll need to use the native Scan end points.

Types of Scan

The security API provides endpoints for different threat object types e.g. file, url and email.

Each has a score, a quickscan and a deepscan endpoint.

These are designed to give a choice between depth of information vs. the latency and cost to generate it as follows:

  • score: Use when the only thing that is needed is a fast & efficient verdict on the threat object. High volume, mostly clean. Lowest cost. Audience is normally code, the vast majority of score responses are clean and not seen by a human. Fast and efficient.
  • quickscan - Use when more contextual information is needed. Medium volume, perhaps more suspicious. Medium cost. The results may be seen by a human - typically an end user (e.g. a detection name in a pop up) or an admin (a quarantine view) its ok to spend a bit longer / pay a bit more.
  • deepscan - Use to throw the book at it. Forensics. Audiences are always human, typically for the admin or SOC person, expert, admin. Highest cost.

As an example design pattern - you may call score, and then if it’s suspicious, do quickscan, then if its malware do a file/deepscan to present the report to the user.

You shouldn’t think of some of the endpoints being synchronous vs. asynchronous and some being block lists vs. scan requests.

All of the end points may return asynchronous results, but it’s more likely to get a synchronous result back from a score endpoint - this is because a verdict can often be generated very quickly, but not always.

All of the file and email endpoints may require an upload of the object but it is more likely that the score endpoint will not need it (for example if it uses a block list to return the verdict).

The scan and score endpoints can and should (but is not mandatory) also contain a scancontext parameter. See the Context Data section for more information.

File Scanning

The file analysis endpoints are below. They all take a SHA256 of the file. See the section on uploading objects to understand when and how files are uploaded.

End Point Description
GET​/file​/score Gets a threat score for a file
GET/file​/quickscan Get a score and a quick threat analysis report for a file
GET/file​/deepscan Get a score and a deep threat analysis report for a file

See File Scanning End Points

Web Scannning

The web analysis endpoints are below. They all take a url which must include the scheme (e.g. https or http).

End Point Description
GET/web​/score Get a threat score for a url
GET/web​/quickscan Get a score and a quick threat analysis report for a url
GET​/web​/deepscan Get a score and a deep threat analysis report for a url

See Web Scanning End Points

Email Scanning

The email analysis endpoints are below. They all take an email_id. An email_id is a user defined unique id of the RFC822 email. It can be the sha256 of the email but it does not have to be. It is up to the caller to ensure uniqueness. Emails have the same object uploading paradigm as files. See the uploading objects section to understand how and when emails should be uploaded.

End Point Description
GET/email​/score Get a threat score for an email mime stream
GET/email​/quickscan Get a score and a quick threat analysis report for an email
GET/email​/deepscan Get a score and a deep threat analysis report for an email
GET/email​/attachmentscan Get a score and a quick threat analysis report for the attachments
GET/email​/bodyscan Get a score and a quick threat analysis report for the body

See Email Scanning End Points

Uploading a Threat Object

The file and email APIs may require the file or email for analysis. If the platform does not have a cached copy of the object and it is needed to complete the request a 404 not found will be issued. The client needs to upload the object and retry to get the results of the analysis.

A client that only wants to use the APIs as a lookup may choose not to upload the full object and proceed appropriately treating the 404 as an ‘unknown’.

Uploading uses the AWS s3 pre-signed paradigm to upload data. This means the data is fielded by AWS s3 infrastructure rather than the AWS Gateway that the security API is exposed.

To upload an object use the GET upload APIs to retrieve a pre-signed post to upload to. There is one for email and file.

End Point Description
GET/file​/upload Upload a file so it can be analysed
GET/email​/upload Upload a email so it can be analysed

See the sample code for working examples of how to do this.

Retrieving Results

The security APIs are all asynchronous in that they will return a report id that can be polled to receive the results. After making the GET call the client should then poll the report API until the status is successful to retrieve the results.

Below is the report endpoint

End Point Description
GET/report​/{report_id} Get the report for the specified report id

It may be that you also get a score and a repot_json in the initial response if the platform was able to complete the request synchronously (e.g. from cache) but your implementation should expect to handle the asynchronous case.

You only need to wait for the result if you are going to act on it (block, warn, prompt etc) - i.e. a client that does not act on the verdict does not need to wait to retrieve the full report instead the report id can be saved and formatted into a portal url for the user to see the report at a later time. If the user accesses that url before the report is ready the portal will handle the waiting experience.

A client that has very low latency requirements may decide to timeout on waits and fall into a follow up non blocking mode for threat objects that are not cached or don’t have a fast response.

Providing Context Data

Context data is passed up with a scan request to provide context. This may be useful for inline analysis and later data mining in the activity store.

Arbitrary name value pairs can be sent but the following our current standards

Key Value
plugin_version One to four dot-separated integers identifying the version of the plugin e.g. ‘3.1.2.4567’ or ‘2.8’
client_uuid A unique RFC4122 v4 uuid for the client generated at install time e.g. “ae1b7a04-4af3-4569-b132-8c1528cb34e8”
file_name The file name of the scan object (file or email) e.g. ‘notepad.exe’
subject The email subject if scanning an email e.g. ‘RE: urgent document’

You should provide the context on the GET request of the security API call. Create a json object with the keys and values you want to pass, convert to a url encoded string and set is as the value for the scancontext url parameter. See the code samples to see how this is done.

Client UUID

This is an rfc4122 compliant unique id that should be created at installation time.

It is used to identify a unique client installation for example to distinguish between plugins use the same api_key in telemetry. For example, it is used to filter scanning history for a specific client in the view scan history portal page.

Example:

‘6491f6d0-40b5-11eb-b378-0242ac130002

Version

This is the version of the client.

Use same the format as https://developer.chrome.com/docs/extensions/mv2/manifest/version/

Example:

2.5.6 or 1.1.456.789

Interpreting Results

This section provides a guide to the json return values from the endpoints.

Result Fields

When a score or can analysis endpoint is called it will return the following fields

Value Description
status The status of the request, either ‘success’, ‘pending’, ‘failure’
threat_object Object containing the threat object for analysis (e.g. { “url” : “https://www.a.com”}
scan_context Object containing the data supplied in scancontext param
datetime The epoch time the request started (e.g. 1608290649.0249567)
account_id The account that will be debited and that the api_key belongs to
api_key The API key used for the request
meta_data Object with with additional data (vendors, credit, client info etc)
report_id The id of the report. This is a cryptographically strong ID

The most important values to the client are the status and the report id, the other values are informational and subject to change.

The report id is the unique id that the client should use to poll the report endpoint to check progress and get the results when completed.

If the status is ‘success’ then the call was satisfied synchronously the response will also contain the fields returned by the report endpoint. However the report endpoint can still be called to retrieve the data again.

If the status is ‘pending’ the report id should be used to poll the report endpoint for completion and results

If the status is ‘failure’ the analysis failed and there should be additional information on the error in the json response

Scan Status

When polling the report endpoint the client will see the following top level values in the json response alongside the values already seen in the submission endpoints.

Value Description
status The status of the request, either ‘success’, ‘pending’, ‘failure’
score The score of the threat score of the object
verdict The verdict (based on the score)
duration The duration of the analysis
json_report Only for the ‘scan’ endpoints. This data is vendor specific and renderable by portal only.

Status is the flow control value, score and or verdict are actionable, duration is informational and json_report should not be parsed by the client. Other metadata fields will also be present but are not meant for plugin use.

If the status is ‘success’ then the above fields will be available and the client can proceed accordingly.

If status is pending the plugin should poll again.

If the status is ‘failure’ the plugin should invoke error handling.

HTTP Codes

Swagger defines the error codes in more detail but the key HTTP return codes are here

Code Description
403 The API key was not valid or the endpoint does not exist
429 The API key has exceeded quota or run out of credit
200 Everything is ok.
404 File or Email not found. Typically it needs uploading.
4xx Some error on the request side
5xx Some error on the server side - look for json body for vendor error

Note: 404 in the above is not an error case and can be the normal part of a file or email analysis flow where the object has not been uploaded before.

Score & Verdict

When an analysis is complete all the APIs will return a threat score and verdict.

The score is in a range from 0-100 and the verdict is a mapping of that score to sub ranges as per the table below.

Range Verdict Description
0-20 Malware Strong reason to believe it is malicious client should block and or alert.
21-40 Suspicious Reason to believe, it’s dangerous or unwanted (PUA/ SPAM). Plugin should block or warn.
41-60 Untrusted No reason to believe malice but the analysis tried and failed to establish trust
61-80 Clean No reason to believe malice. This is the default fail safe value.
81-100 Trusted The analysis was able to establish trust (certificate or trust list)

Some of the simplest scan configurations will not try and establish trust and so results will only be in the malware or clean ranges. Others have more fine grained outputs and map to the full range.

The fail policy is fail open and failures will map to the clean range.

Spam and configurations that distinguish files as PUA/PUP will land in the suspicious range.

Configurations that identify phish emails or urls will land in the malware range.

Scanner Report

The Score endpoints only return a score and verdict, these can be interpreted and actioned directly by the client.

But the scan endpoints also respond with scanner specific report data serialised in the json_report object. This data is scanner specific and subject to change, a client should not attempt to parse it. Instead redirect a user to the portal landing urls to display the reports. The portal has vendor specific PHP twig templates that are kept up to date and can render vendor specific details.

Portal Redirect URLs

To integrate with the portal your client can redirect the user to the following urls where appropriate.

The URLs can take parameters as GET or POST with the later being more secure. The pages will then redirect to the appropriate material in the portal.

View Report

Render a report give an report_id |Name|Value| |— |— | |Path|https://secplugs.com/plugin_landing/viewreport.php| |report_id|The report id to render|

View Scan History

Show scan activity history for the specified client |Name|Value| |— |— | |Path|https://secplugs.com/plugin_landing/viewscanhistory.php| |client_uuid|The client uuid generated by the client at installation time| |api_key|Current (default, anonymous) api key in use. Portal will match plugin type.|

Detection / Warning

Detection / Warning - Given a report_id show a detection / warning page |Name|Value| |— |— | |Path|https://secplugs.com/plugin_landing/showalert.php| |report_id|The report id to render|

Sign Up

Sign Up - Start the user flow to sign up and become a registered user |Name|Value| |— |— | |Path|https://secplugs.com/plugin_landing/signup.php| |client_uuid|The client uuid generated by the client at installation time| |api_key|Current (default, anonymous) api key in use. Portal will match plugin type.|

Sign In

Direct the user to the sign in page |Name|Value| |— |— | |Path|https://secplugs.com/plugin_landing/signin.php|

Visit Us

Direct the user to the web site |Name|Value| |— |— | |Path|https://secplugs.com/plugin_landing/visitus.php|

Get API Key

Start the user flow to get an api key for this client |Name|Value| |— |— | |Path|https://secplugs.com/plugin_landing/getapikey.php| |client_uuid|The client uuid generated by the client at installation time| |api_key| Current (default, anonymous) api key in use. Portal will match type.|

Learn More

Take the user to pages about the client and benefits of signing up |Name|Value| |— |— | |Path|https://secplugs.com/plugin_landing/learnmore.php| |client_uuid|The client uuid generated by the client at installation time| |api_key|Current (default, anonymous) api key in use. Portal will match type.|

Testing Your Client

Sections describes tools to help test your client.

Mock API Keys & Test Points

Within the platform there is a mock scanner implementation created for testing.

The mock implementation has the following behaviour:

  • File APIs will report a malware verdict ( score = 10 ) if EICAR is used
  • Email APIs will report a suspicious verdict ( score = 30 ) if a GTUBE is used
  • The Web APIs will return a malware verdict score ( score = 10 ) if the url contains example.com/test_malware
  • The Web API will return a suspicious verdict ( score = 30 ) if the url contains example.com/test_suspicous
  • Everything else will return clean (score = 70 )

There are two mock API Keys that are configured to use the mock vendor implementation as follows:

For synchronous Testing use

P7QnYz39Vo6BfHY6P8QvFapwptrto0CyaflSZHGW

For asynchronous Testing use

GW5sb8sj8D9CtvVrjsTC22FNljxhoVuL1UoM6fFL

The synchronous key will return results immediately where as the asynchronous key will return a pending result until the 5th call when it will return the mock result.

Note: The above keys are public domain and do not need to be kept secret. They are protected from abuse with usage quotas.

Test Points

There is a variety of test samples (emails, files and urls) on the Secplugs portal below

https://secplugs.com/index.php/test-sample-page

These include the Test Points for use with the mock Api Keys, industry standard test points and some real zoom malware.