Ga naar inhoud

Logging and audit requirements

These requirements cover the three records of GBO: the decision record (ADL), the processing record (LDV), and the trace that links them. For the design, see Logging, audit and traceability.

This page has the following parts:

Decision audit requirements

The decision audit requirements cover the record of authorization decisions. They set rules for these topics:

  • the Authorization Decision Log (ADL) of Logius
  • the protection of the record
  • the policy version that the record names
  • the personal data that the record can hold

The requirements are generic. They apply to DvTP, EUDI issuance, and the ASI route. Processing logging covers the record of each processing (LDV). Observability covers the trace ID.

REQ-32: Audit records are protected, complete, and stored apart from operational logs

Decision records (ADL) and processing log records (LDV) MUST be protected against change and deletion. This protection applies to anyone who does not manage them. They MUST NOT be sampled. They MUST have their own retention period and access, independent of those of operational logs and traces. Operational logs MAY carry the same trace ID, but they are no audit source.

  • Status: ready for review
Source, evidence and notes
  • Source: design:
    • PSA S09: measures protect log data against unauthorized change and deletion, and agreements set retention, access, inspection, and destruction explicitly ("maatregelen beschermen loggegevens tegen ongeautoriseerde wijziging en verwijdering", "afspraken regelen bewaartermijnen, toegang, inzage en vernietiging expliciet").
    • LDV, behavior of the application (no log sampling).
    • P-12.
  • Addressee: GBO-delivered component: the logbook and the components that write to it. Interface and audit agreement: the decision record at the source.
  • Evidence:
    • The ADL is in Postgres, apart from the console log in Loki (docker-compose.yml#L117-L125, PR #337).
    • Readers get only SELECT (adl-reader.sql#L17, PR #373).
    • The logbook in the demo has write and read, and no change or delete (handler.go#L49-L53). That logbook is a stand-in for the logbook of the controller (Verantwoordelijke). It is no GBO component.
    • Retention period and tamper resistance at storage level: not built.
  • Related: REQ-71, REQ-72, REQ-123, Q-08, ADR 0036.
  • Notes: retention periods and read access are open in Q-08. Nobody tested whether the writing role of the PDP can change its own records.

REQ-59: The decision record names the policy version that decided

Every decision record (REQ-71) MUST carry the identity and version of the policy set that made the decision (adl.core.policies). For every policy set, it MUST be traceable which version became active when. The source decides how it forms that version (bundle ID with version, commit hash, image version).

  • Status: ready for review
Source, evidence and notes
  • Source: design:
    • PSA S09: authorization decisions refer to the applied policy version and to relevant information about the legal basis ("autorisatiebeslissingen verwijzen naar de toegepaste beleidsversie en relevante informatie over de grondslag").
    • ADL adl.core.policies.
    • PSA S06 (start date and change history).
  • Addressee: interface and audit agreement: every PDP that decides on a data request.
  • Evidence: not built, and there is no demo issue for it. The bundle carries a version (gbo-pdp.yaml#L17), but that version does not reach the ADL record (PR #373). Records carry policies: 0.
  • Related: REQ-71, F-53, Q-08, Q-21, RL-01, RL-04.
  • Notes: without this field, nobody can link the history of the policy (who published which version) to a decision. In the default stack, the policy comes from the PDP image. In that case, the version must also be traceable.

REQ-60: Logging contains no BSN that the processing does not need

A GBO-delivered component MUST keep the BSN and other directly identifying data out of its operational logs and spans. Some identifying values in the PDP input are not necessary for the decision. A component that builds the PDP input MUST keep such a value out of the input, or replace it before it sends the input. If that fails, the request does not continue. Masking in the decision log MUST NOT be the only safeguard.

Case Outcome
Operational log or span no BSN. A data subject (Betrokkene) only as pseudonym or opaque reference
PDP input, DvTP no identity: a placeholder, which the source replaces only after the consent
Decryption or replacement fails the request does not reach the source
PDP input, EUDI and ASI no BSN (not met in demo v0.9.0, see the notes)
  • Status: ready for review
Source, evidence and notes

REQ-71: Every authorization decision is in an ADL-conformant decision record, and that record is the audit source

Every authorization decision on a data request MUST be recorded in a record according to the Logius standard Authorization Decision Log. The record MUST make it possible to replay the decision. It holds the request, the answer, and the policy version (REQ-59). It also holds the answer of every information source consulted during the evaluation (adl.core.information). A reference to that answer can replace the answer. The reference MUST then stay available as long as the record exists.

If the request passes an FSC Inway or Outway, adl.fsc.transaction_id MUST be set. Where more decision logs exist side by side, the ADL record MUST be the designated audit source. Other streams MUST NOT be an audit source.

Case Outcome
Allowed or denied one record, with outcome and reason code
Evaluation fails also one record
Decision depends on the consent status the status answer is in or with the record
Request through FSC adl.fsc.transaction_id next to trace_id
  • Status: ready for review
Source, evidence and notes
  • Source: design:
  • Addressee: interface and audit agreement: the PDP of every source, whatever the product. GBO-delivered component: what GBO delivers to the PDP (request mapping, policy).
  • Evidence: the ADL in Postgres is mandatory (docker-compose.yml#L125, PR #337). The demo reads it as the audit source (PR #373). The demo does not meet these parts:
    • transaction ID and trace behind the Inway (#369)
    • policy version (REQ-59)
    • status answer in the record (F-72)
  • Related: REQ-32, REQ-55, REQ-59, REQ-60, REQ-06, F-41, F-53, F-63, F-72, Q-08, Q-25, ADR 0036.
  • Notes:
    • The detail for each field (which fields were allowed or denied) does not need to be in the record. It follows from a replay, and the ADL prefers a reference to a copy.
    • A reference brings a retention duty for its source (Q-08).
    • The ADL is working version 1.0.0. Pin the version in a conformance claim.
    • The policy fetches the consent status during evaluation. For that reason, a replay from (input, policy, data) needs the status answer in the record.

Removed numbers

  • REQ-31 (persistent, correlatable audit record for each authorization decision): merged into REQ-71.

Processing logging requirements

The processing logging requirement covers the record of each processing of personal data. Decision audit covers the decision side (ADL). The requirement on this page covers the processing side. It is generic and applies to DvTP, EUDI issuance, and the ASI route.

The Logius standard Logboek Dataverwerkingen (LDV) asks for a log record for every processing, for each controller. Only trace metadata crosses an organization boundary. The decision of a PDP is no processing. A transformation at the source, such as the decryption by the source gateway, is a processing (F-64).

REQ-72: A GBO component that processes personal data writes an LDV log record for each processing at its controller

Every GBO-delivered component that processes data of a data subject MUST write a log record according to Logboek Dataverwerkingen for each action. It MUST write the record to the logbook of the controller of that processing. The trace ID in that record MUST follow from REQ-55. There MUST be a separate log record for each data subject. A further data subject of the same processing hangs as a child under the first.

The field dpl.core.data_subject_id MUST name the data subject in an identifier space of the controller itself, with an explicit data_subject_id_type. It MUST NOT name the data subject with the BSN. The field dpl.core.processing_activity_id MUST refer to a versioned, resolvable processing activity. A component that calls another party during the processing MUST set dpl.read.nextLogbookId to the read API of that party. Only trace metadata crosses an organization boundary.

  • Status: ready for review
Source, evidence and notes
  • Source: design:
    • LDV and its read extension.
    • PSA S09 (LDV as functional frame, and each component records the events that are relevant for its responsibility: "iedere component legt de gebeurtenissen vast die voor zijn verantwoordelijkheid relevant zijn").
    • GO, reused building blocks.
    • FR-31 criterion 1.
  • Addressee: GBO-delivered component: source gateway (bron-sidecar), consent register (toestemmingsregister), consent portal (toestemmingsportaal), EUDI adapter, and ASIP. The logbook and the source application of a bronhouder belong to that bronhouder (P-10).
  • Evidence: ICTU/GBO-demo#302, PR #366, PR #469. Code and tests:
  • Related: REQ-32, REQ-55, REQ-60, REQ-76, AV-02, AV-10, F-64, F-68, F-69, Q-08, Q-25, Q-72, ADR 0026.
  • Notes:
    • The ldv-logboek in the demo is a stand-in for the logbook of the controller, and is no GBO component. It refuses a record with a BSN (validate.go#L148, tested in validate_test.go#L94) or with an activity that does not resolve (validate.go#L119). That is therefore no evidence for this requirement.
    • LDV allows BSN as a type. "Never the BSN" is a stricter rule from S09 and P-04.
    • The register of processing activities is a stand-in for each logbook (register.go#L28). A real register is missing. For GBO components it belongs with the management organization (Q-08).
    • dpl.core.foreign_operation.processor is a URL made from the FSC peer ID or the grant hash. It identifies the caller, but nothing resolves behind it.
    • In demo v0.9.0, the ASIP writes no log records.
    • Design choice: a processing delivers data only after its log record is durable locally (outbox). An outage of the logbook does not stop the chain (graphql-server/main.go#L394-L398, bron-sidecar/main.go#L541-L549).
    • It is open whether GBO components write attributes outside the LDV core: Q-72.

Removed numbers

  • REQ-73 (a consumer sends trace context and records the transaction ID of every source call): removed, replaced by connection condition AV-01.

Observability requirements

The observability requirement covers the correlation of logs and traces over the chain. Each request has one trace ID, and the components record the FSC transaction ID next to it. The requirement is generic. It applies to DvTP, EUDI issuance, and the ASI route. Decision audit covers the decision record. Processing logging covers the processing record.

REQ-55: One trace ID per request across the whole chain, and the transaction ID can be linked to it

Every GBO-delivered component MUST take the trace ID of an incoming request unchanged from W3C Trace Context (traceparent). It MUST record the span of the caller as parent. It MUST send traceparent with every outgoing call. Every record that the component writes about the request (processing log record, operational log record, span) MUST carry that trace ID. The Fsc-Transaction-Id MUST NOT replace an incoming trace. A component that starts or receives an FSC transaction MUST record the transaction ID next to the trace ID.

Case Outcome
Request with traceparent trace ID taken over, and the span of the caller is parent. An Fsc-Transaction-Id that comes with it does not change the trace ID
Request without traceparent, with Fsc-Transaction-Id the transaction ID MAY be the trace ID
Entry component without traceparent that starts a transaction itself MAY choose its trace ID equal to that transaction ID
Processing with more than one FSC hop one trace ID, several transaction IDs. The link runs through the records that carry both, and does not depend on equality
  • Status: ready for review
Source, evidence and notes

Removed numbers

  • REQ-16 (end-to-end traceparent propagation): merged into REQ-55.
  • REQ-28 (logs and traces can be reconstructed by a shared identifier for each request): merged into REQ-55.
  • REQ-29 (no BSN in observability logs): merged into REQ-60.
  • REQ-54 (peer transaction log and portal correlation by Fsc-Transaction-Id): merged into REQ-55.