24 Feb, 2020

The Quttera Website Malware Scanner API

At Quttera, we're constantly developing and integrating new cybersecurity software and services. In this article, we'll look at our new website malware scanner API.
At Quttera, we're constantly developing and integrating new cybersecurity software and services. Our growing product line provides a comprehensive range of instruments to IT professionals and security companies for fighting cybercrime. An efficient approach to protecting Web assets needs to cover the following aspects:
  • External scanning
  • Server-side scanning
  • Intrusion detection with high quality and precision
  • Removal of malware infections
  • Ongoing website protection

Our proprietary, patented portfolio includes external Web malware scanners based all over the world, server-side Web malware scanners, automated website malware cleanup, and endpoint and DNS-based Web application firewalls (WAFs).

This post provides a short introduction to the REST API to help anyone seeking to use it with their monitoring applications, security operations centers (SOC), or other appliances to scan websites for malware. The focus is on using the Quttera Malware Scanner API feature to facilitate an HTTP-based scan of a domain or URL.
What is a REST API?
REST stands for "Representational State Transfer." It is a software architecture and design approach built on the HTTP protocol. A REST query is an HTTP request which satisfies certain conditions. A Web service that supports the REST API is called a RESTful service.

HTTP supports four methods: GET, POST, PUT, and DELETE. Browsers rarely use the last two, but REST uses all four of them. A REST API defines a set of HTTP methods in the form of URI paths. The path specifies the resource to operate on and may include parameters as part of the path.

The API client creates and sends the requests, and the API server executes appropriate logic to carry them out. The server may perform authentication, such as checking an API contained in the request. Requests are stateless; each request contains all the information needed to carry it out, without using cookies.

The response body may contain data in any format. Many RESTful services, including Quttera's, give their responses as JSON data.
Here is a very simple example of a REST API for creating, deleting, and obtaining information on users.

HTTP POST https://apiserver.com/user # create user
HTTP GET https://apiserver.com/user # to retrieve information on a user
HTTP DELETE https://apiserver.com/user # remove a previously created user

How the Web Malware Scanner API works
The Quttera Web Malware Scanner API defines four queries for scanning a website or URL.
  • scan: Start an investigation of a specified URL or domain.
  • status: Check the status of a previously launched scan.
  • report: Retrieve a report from a completed scan on a URL or website.
  • blacklist_status: Check whether a given domain name is present in the Quttera blacklist database.
The following pseudocode shows how the scan procedure works on the client-side:

scan(URL) # start the scan of the given URL
while status(URL) is not DONE: # wait for the server to finish investigation  
    sleep (1 second)
report(URL) #retrieve the investigation report from the API server

Python code to perform the scan looks like this:

url = "http://quttera.com"
response = requests.post("http://apiserver.quttera.com/api/v3/my-api-key/url/scan/{}".format(url))
status = None
while status is not "done":
   response = requests.get("http://apiserver.quttera.com/api/v3/my-api-key/url/status/{}".format(url))
   st = response.json()
   status = st["status"]["state"].lower()
response = requests.get("http://apiserver.quttera.com/api/v3/my-api-key/url/report/{}".format(url))
report = response.json()

This is a simplification. The “/” characters in the URL will mess up the request if passed unencoded. It needs to be encoded using the Python Base32 or Base64 algorithm. Here is an example using the encoding.

url = base64.b64encode("http://quttera.com")
  # or base64.b32encode("http://quttera.com") for base32 encoding
response = requests.post("http://apiserver.quttera.com/api/v3/my-api-key/url/scan/{}".format(url))
status = None
while status is not "done":
    response = requests.get("http://apiserver.quttera.com/api/v3/my-api-key/url/status/{}".format(url))
    st = response.json()
    status = st["status"]["state"].lower()

response = requests.get(http://apiserver.quttera.com/api/v3/my-api-key/url/report/{}".format(url))
report = response.json()
The investigation report will contain:
  • The blacklist report for the URL.
  • The blacklisting status for all links extracted from the scanned content.
  • A list of all scanned files and their classification.
For a more detailed description of the API scan report, see the Website Malware Scanner REST API online help. The help page provides additional information on request parameters, data formats, and return codes.
Getting the blacklist status using the API
Sometimes, rather than a full scan, just a URL blacklist status is necessary. The scanner API provides the "/blacklist/status" query for that purpose.
The following Python code shows that use:

response = requests.get("http://apiserver.quttera.com/api/v3/my-api-key/blacklist/status/{}".format(url))
status = response.json()["status"]
# supported values: NoThreat, GenericMaliciousObject, GenericSuspiciousObject
if status is "GenericMaliciousObject" or status is "GenericSuspiciousObject":
   echo "URL blacklisted"

What types of malware status does the API detect?
The Quttera Web Malware Scanner API interfaces to our patented external Web malware scanner. The scanner provides highly sensitive heuristic detection capabilities. Its features include JavaScript emulation, penetration modules, pattern matching, and shellcode entropy detection.
The script coder can configure detection levels to score an object for its severity. The basic configuration provides four levels:
  • Clean: The scanned object (HTML, JavaScript, CSS, or text) is safe
  • Potentially suspicious: The scanned object may require manual investigation.
  • Suspicious: The scanned object is probably malicious and requires manual investigation.
  • Malicious: The scanned object contains a malware infection.
The levels can be adjusted to fit a company's website security policy. The number of categories can be increased or decreased, and the content of each category can be changed.
Suspicious and malicious objects can fall into these categories:
  • Hidden malicious iframes
  • Malicious encoded JavaScript
  • JavaScript code injection
  • Malicious redirection in HTML or JavaScript
  • Unconditional redirections
  • Hidden spam
  • Cryptomining code
  • References to blacklisted resources
  • Website defacement
For more information about Quttera's external malware scanners, see our "About Us" page and our press releases.
Quttera Web Malware Scanner API vs. URL reputation APIs
The Quttera Web Malware Scanner goes far beyond URL reputation APIs. A reputation API provides the last known classification of a website, not on-demand scanning of the current content. It only searches a database that holds previously collected information.

For example, if a website was last checked a week ago for the database, a reputation API will present the same result today as it did then. The result won't be updated until the website is rescanned. It's similar to the blacklist search API, providing information on whether the website had malicious content when it was last scanned.

In contrast, our Web Malware Scanner API performs an on-demand scan each time it is invoked. It simulates Web browser behavior and returns the current investigation status. This will include the blacklisting status of every external URL found on the website.
How to integrate the API into an application
The integration of the API into a script or application is straightforward. It just requires an HTTP client library which is compatible with the coding environment. In this article, we presented some simple Python code to scan a single URL or domain. Any programming language can be used to implement the same logic, using an appropriate HTTP client library.
The Quttera public repository has Python and PHP versions of API clients which you can use as a reference for client development:
Where the scanner API can be used
The scanner API is suitable for any application that needs to perform a real-time investigation of a given domain or URL. Our partners who have integrated the API include several CERT organizations, Web agencies monitoring their managed sites, online scanners such as VirusTotal, cybersecurity companies, and Web hosting providers.

To learn more about Quttera's API partnership program and find out how to become a partner, see our Quttera Partners page.