Skip to main content

Post-processing with Hooks

Introduction​

In this step-by-step tutorial, we will go through all the required stages to set up hooks with the secureCodeBox. Hooks can be used to perform post-processing on findings, for which we'll give a few examples.

Hooks Basics​

In principle, the secureCodeBox works in such a way that the raw findings (the result provided by a scanner) are stored in an S3 storage (e.g. Minio). These stored raw findings are converted by specific parsers (each scanner has its own implementation) into a uniform secureCodeBox-specific finding format and will be also stored in the S3 storage. All configured hooks that can read (Read Hooks) and write (ReadAndWrite Hooks) these Findings are then executed. More details can be found in ADR-0002.

The basic purpose is to transfer the findings to third-party systems. Findings can be read with a ReadHook and processed further as required (e.g. saved in another system via a REST call). On the other hand, ReadWriteHooks also offer the option of adding further information to Findings or modifying them.

caution

In principle, both the raw findings and the secureCodeBox findings are available to hooks via API. In principle, the uniformed secureCodeBox findings should be preferred. However, the DefectDojo persistence hook, for example, only processes the raw findings. This means that post-processing hooks that work on the secureCodeBox finding have no effect on the data that ends up in DefectDojo.

This means that depending on what you want to achieve with your hook, you need to look at how the following hooks process the findings.

Setup​

For the sake of the tutorial, we assume that you have your Kubernetes cluster already up and running and that we can work in your default namespace. If not, check out the installation for more information.

We'll start by installing the nmap scanner:

helm upgrade --install nmap secureCodeBox/nmap

Next, we'll install two update-field hooks:

helm upgrade --install ufh1 secureCodeBox/update-field-hook --set attribute.name="category" --set attribute.value="first-hook"
helm upgrade --install ufh2 secureCodeBox/update-field-hook --set attribute.name="category" --set attribute.value="second-hook"

The first hook will update all secureCodeBox findings such that the field category is set to the value first-hook. The second hook will set the same field to second-hook. For a list of all available secureCodeBox hooks, see hooks. There's no limit to the amount of hooks you can install.

Creating a Scan​

In practice, you are not required to specify anything to run your hooks.

apiVersion: "execution.securecodebox.io/v1"
kind: Scan
metadata:
  name: "nmap-example"
spec:
  scanType: "nmap"
  parameters:
    # We'll just scan for port 80 to speed up the scan.
    - "-p"
    - "80"
    # Scanning an example domain
    - "scanme.nmap.org"

For starting scans in Kubernetes, see First Scans.

Once the scan has finished, you will see that two hooks have run on your scan results with following command:

kubectl get pods

which will show something similar to this:

NAME                                                 READY   STATUS      RESTARTS   AGE
parse-nmap-example-5p964--1-ctgrv                    0/1     Completed   0          18s
scan-nmap-example-gg9kd--1-pjltd                     0/2     Completed   0          26s
ufh1-update-field-hook-nmap-example-p8pcb--1-vx25q   0/1     Completed   0          10s
ufh2-update-field-hook-nmap-example-drjmq--1-vzds2   0/1     Completed   0          3s

Inspecting the Findings​

Looking at the findings, you will notice that the category field has been set to second-hook. This happens because the ufh2 hook was executed after ufh1, discarding the value first-hook completely.

[
  {
    "name": "Open Port: 80 (http)",
    "description": "Port 80 is open using tcp protocol.",
    "category": "second-hook",
    "location": "tcp://45.33.32.156:80",
    "osi_layer": "NETWORK",
    "severity": "INFORMATIONAL",
    "attributes": {
      ...
    },
    "id": "9fbda429-478d-4ce0-9a8d-1c4aef4d9b58"
  }
]

Execution Order of Hooks​

By default, hook order is specified according this definition.

With the hook.priority field, you can further customize the order of secureCodeBox hooks. The higher the priority of a hook, the earlier it will execute. By default, all hooks have a priority of 0.

If we set ufh2 hook's priority to 1, we'll observe that it will execute before ufh1.

helm upgrade --install ufh2 secureCodeBox/update-field-hook --set hook.priority="1" --set attribute.name="category" --set attribute.value="second-hook"
kubectl get scancompletionhooks.execution.securecodebox.io
NAME                     TYPE           PRIORITY   IMAGE
ufh1-update-field-hook   ReadAndWrite   0          docker.io/securecodebox/hook-update-field:3.3.1
ufh2-update-field-hook   ReadAndWrite   1          docker.io/securecodebox/hook-update-field:3.3.1

Start, the scan and observe orders:

kubectl get pods                                                                                                kube minikube
NAME                                                 READY   STATUS      RESTARTS   AGE
parse-nmap-example-lrtcl--1-57n9t                    0/1     Completed   0          36s
scan-nmap-example-7s2t8--1-gbr6b                     0/2     Completed   0          39s
ufh1-update-field-hook-nmap-example-cvzw2--1-x4rcz   0/1     Completed   0          30s
ufh2-update-field-hook-nmap-example-mv57q--1-cvd4k   0/1     Completed   0          33s

Kubernetes sorts the list alphabetically, but notice the age of the jobs. Looking at the resulting finding, we can see the category is set to first-hook.

[
  {
    "name": "Open Port: 80 (http)",
    "description": "Port 80 is open using tcp protocol.",
    "category": "first-hook",
    "location": "tcp://45.33.32.156:80",
    "osi_layer": "NETWORK",
    "severity": "INFORMATIONAL",
    "attributes": {
      ...
    },
    "id": "c15d1730-7ca8-4529-b55a-a3412832f309"
  }
]

Hook Selector​

An alternative for more runtime hook control is the scan's HookSelector. This field allows you to define which hooks to run for a scan.

In this case, we select all hooks, except hooks with the label ufh1.

apiVersion: "execution.securecodebox.io/v1"
kind: Scan
metadata:
  name: "nmap-example"
spec:
  hookSelector:
    matchExpressions:
    - key: app.kubernetes.io/instance
      operator: NotIn
      values: [ "ufh1" ]
  scanType: "nmap"
  parameters:
    # We'll just scan for port 80 to speed up the scan.
    - "-p"
    - "80"
    # Scanning an example domain
    - "scanme.nmap.org"

You can find that only ufh2 was executed.

kubectl get pods
NAME                                                 READY   STATUS      RESTARTS   AGE
parse-nmap-example-shkrr--1-2hdt9                    0/1     Completed   0          10s
scan-nmap-example-7bllp--1-zx287                     0/2     Completed   0          13s
ufh2-update-field-hook-nmap-example-lmljv--1-smgp5   0/1     Completed   0          7s

The following labels are available by default:

  • app.kubernetes.io/instance: the Helm release name (e.g. ufh1, ufh2)
  • app.kubernetes.io/name: the Helm chart name (e.g. update-field-hook)
  • securecodebox.io/internal: boolean field for whether this hook has internal usages in secureCodeBox (e.g. Cascading Scans hook)

You can also deploy secureCodeBox hooks with your own labels like so:

helm upgrade --install ufh2 secureCodeBox/update-field-hook --set hook.labels.securecodebox="rocks" --set attribute.name="category" --set attribute.value="second-hook"

This will add your custom label to the secureCodeBox hook so that you can select is with hookSelector.

apiVersion: execution.securecodebox.io/v1
kind: ScanCompletionHook
metadata:
  labels:
    app.kubernetes.io/instance: ufh2
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: update-field-hook
    helm.sh/chart: update-field-hook-3.3.1
    securecodebox: rocks
  name: ufh2-update-field-hook
  namespace: default
spec:
  env:
  - name: ATTRIBUTE_NAME
    value: category
  - name: ATTRIBUTE_VALUE
    value: second-hook
  image: docker.io/securecodebox/hook-update-field:3.3.1
  ttlSecondsAfterFinished: null
  type: ReadAndWrite

Cascading Scans​

The HookSelector field is also available in Cascading Rules. This means that you can selectively disable hooks for certain rules. Let's say that you're running secureCodeBox with nmap, ncrack, and a DefectDojo persistence provider. We can imagine that you'd prefer your ncrack passwords to not go directly to DefectDojo, so you could set up the cascading rule such that it filters the DefectDojo hook.

apiVersion: "cascading.securecodebox.io/v1"
kind: CascadingRule
metadata:
  name: "ncrack-ftp"
  labels:
    securecodebox.io/invasive: invasive
    securecodebox.io/intensive: high
    securecodebox.io/type: bruteforce
spec:
  matches:
    anyOf:
      - category: "Open Port"
        attributes:
          service: "ftp"
          state: open
  scanSpec:
    hookSelector:
      matchExpressions:
      - key: app.kubernetes.io/name
        operator: NotIn
        values: [ "persistence-defectdojo" ]
    scanType: "ncrack"
    parameters:
      - -v
      - -d10
      - -U
      - /ncrack/users.txt
      - -P
      - /ncrack/passwords.txt
      - -p
      - ftp:{{attributes.port}}
      - "{{$.hostOrIP}}"

Note that we use app.kubernetes.io/name here to filter all releases of the DefectDojo persistence provider.

caution

You can use scan.spec.cascading.inheritHookSelector on your initial scan definition to pass hookSelector entries on to cascaded scans. Selectors defined in cascading rules will only apply to the scan triggered by the rule - if the results of that scan then trigger further cascading scans, the selectors defined in the cascading rule will be dropped and only those from the original scan are kept. Defining identical entries in both the Scan AND the Cascading Rule resource will lead to undefined behaviour.

See #789 for more details.