Skip to content
Toposort Logo
Knowledge Base

Event Collection Methodology

Overview

This document outlines Topsort’s methodology for collecting, validating, and processing advertising events in compliance with IAB/MRC Retail Media Measurement Guidelines. All event data flows through standardized collection pipelines with built-in quality controls and validation.

Event Types Collected

Topsort collects four primary event types:

  1. Impressions - User viewed an ad asset (listing or banner)
  2. Clicks - User clicked on an ad asset
  3. Purchases - User completed a transaction
  4. Page Views - User visited a page (for context and analytics)

All events require either a resolvedBidId (for sponsored events from auction responses) or an entity descriptor (for organic events).

Collection Methods

API-Based Event Reporting

Primary collection method using the /events endpoint:

{
  "impressions": [{
    "id": "event_uuid",
    "resolvedBidId": "auction_response_token",
    "placement": {
      "path": "/category/shoes",
      "location": "top_banner"
    },
    "occurredAt": "2024-12-01T10:30:00.000Z",
    "opaqueUserId": "user_hash_value"
  }],
  "clicks": [{
    "id": "event_uuid",
    "resolvedBidId": "auction_response_token",
    "placement": {
      "path": "/product/12345"
    },
    "occurredAt": "2024-12-01T10:31:15.000Z",
    "opaqueUserId": "user_hash_value"
  }],
  "purchases": [{
    "id": "event_uuid",
    "occurredAt": "2024-12-01T10:45:00.000Z",
    "opaqueUserId": "user_hash_value",
    "items": [{
      "productId": "sku_12345",
      "unitPrice": 49.99,
      "quantity": 1
    }]
  }]
}

Data Capture Requirements

Each event type captures specific required and optional fields:

Impressions:

  • Required: resolvedBidId or entity, occurredAt, opaqueUserId
  • Optional: placement, viewability data, device info

Clicks:

  • Required: resolvedBidId or entity, occurredAt, opaqueUserId
  • Optional: placement, additionalAttribution

Purchases:

  • Required: occurredAt, opaqueUserId, items array
  • Optional: resolvedBidId (for direct attribution), orderId

Data Flow Pipeline

  1. Event Submission

    • Client submits event via POST to /events endpoint
    • Request authenticated with API key
    • Timestamp recorded at server ingestion

  2. Schema Validation

    • JSON structure validated against event schema
    • Required fields presence verified
    • Data types and formats checked

  3. Real-time Quality Checks

    • SIVT detection algorithms applied
    • Duplicate detection (by event ID)
    • Timestamp validation (within acceptable window)

  4. Enrichment

    • Auction data joined via resolvedBidId
    • Campaign and vendor metadata attached
    • Geographic and device data normalized

  5. Storage

    • Raw events stored in event log (90 days retention)
    • Validated events written to attribution database
    • Aggregated metrics updated in reporting datastore

  6. Processing

    • Attribution calculations executed
    • Metrics aggregated by campaign/product/vendor
    • Reports updated for dashboard access

Validation and Quality Controls

Timestamp Validation

Events are validated for temporal consistency:

  • Future events: Rejected if occurredAt is more than 5 minutes in future
  • Past events: Accepted up to 7 days in past for late-arriving data
  • Clock skew: Server timestamp compared to client timestamp with tolerance
  • Out-of-order: Events processed with consideration for delivery order vs occurrence order

Duplicate Detection

Prevention of duplicate event counting:

// Event deduplication by UUID
{
  "id": "550e8400-e29b-41d4-a716-446655440000",  // Client-generated UUID
  "resolvedBidId": "...",
  "occurredAt": "2024-12-01T10:30:00.000Z"
}
  • Event IDs must be unique UUIDs
  • Duplicate IDs within 24-hour window rejected
  • Idempotent processing ensures same event not counted twice

Attribution Token Validation

All sponsored events validated against auction records:

  1. resolvedBidId decoded and verified against auction database
  2. Auction expiration checked (typically 24-hour validity)
  3. Campaign status verified (active at time of event)
  4. Bid amount and auction parameters retrieved for billing

Data Sources and Identity

User Identity

User identification via:

  • opaqueUserId: Hashed user identifier provided by marketplace
  • No PII: Topsort never receives raw personal information
  • Consistent hashing: Marketplace must use stable hashing algorithm
  • Cross-session: Same user must receive same hash across sessions

First-Party Data

All event data is first-party:

  • Reported directly by marketplace via API
  • No third-party pixels or tags
  • No data brokers or panel extrapolation
  • Direct measurement only

Event Attribution Logic

Attribution Windows

Default windows applied unless customized:

  • Click attribution: 14 days
  • View attribution: 1 day
  • Multi-touch: Last click wins (configurable to last impression)

Conversion Matching

Purchase events matched to ad exposures:

IF purchase.opaqueUserId == click.opaqueUserId
AND purchase.occurredAt - click.occurredAt <= attribution_window
AND purchase.items contains product from campaign
THEN attribute conversion to campaign

Multiple touchpoints handled via configured attribution model (last-click, last-impression, or position-based).

Privacy and Compliance

Data Handling

  • Pseudonymization: All user IDs are hashed before transmission
  • No tracking: Topsort does not place cookies or trackers on end-user devices
  • Marketplace control: Marketplace owns all user data and identity
  • Data minimization: Only required fields collected

Retention Policies

Data TypeRetention PeriodPurpose
Raw event logs90 daysDebugging and SIVT analysis
Aggregated metrics24 monthsReporting and analytics
Campaign data24 monthsHistorical performance
SIVT detection logs180 daysAudit and appeals

Compliance Standards

Event collection meets:

  • GDPR - Lawful basis via marketplace’s data processing agreement
  • CCPA - No sale of personal information
  • IAB/MRC - Transparent methodology disclosure
  • SOC 2 Type II - Security and availability controls

Error Handling

Event Rejection

Events may be rejected for:

  • Schema validation failures
  • Invalid authentication
  • Expired resolvedBidId
  • Future timestamp beyond tolerance
  • SIVT detection

Rejected events return HTTP 4xx with detailed error message.

Partial Success

Batch submissions processed atomically per event type:

  • Valid events accepted
  • Invalid events rejected with specific errors
  • Response indicates success/failure per event
{
  "impressions": {
    "accepted": 98,
    "rejected": 2,
    "errors": [
      {
        "id": "event_123",
        "reason": "Invalid resolvedBidId"
      }
    ]
  }
}

Audit and Verification

Access to Event Logs

Marketplaces can request:

  • Sample of raw event logs for specific campaigns
  • Event-level detail for specific user journeys
  • Attribution calculation breakdowns
  • SIVT filtration decisions

Third-Party Verification

For MRC accreditation:

  • Independent auditors have access to event collection code
  • Sample event logs provided for verification
  • Validation logic auditable
  • Audit trail maintained for all changes

Technical Specifications

API Endpoint

POST https://api.topsort.com/v2/events
Content-Type: application/json
Authorization: Bearer {marketplace_api_key}

Rate Limits

  • 1,000 requests per second per marketplace
  • 10,000 events per batch request
  • Automatic retry with exponential backoff recommended

Latency Targets

  • p50: < 50ms response time
  • p95: < 200ms response time
  • p99: < 500ms response time

Best Practices for Marketplaces

Implementation

  1. Generate Unique Event IDs

    • Use UUID v4 for all events
    • Never reuse event IDs
    • Client-side generation recommended

  2. Batch Event Submission

    • Collect events client-side
    • Submit in batches every 5-30 seconds
    • Include multiple event types in single request

  3. Handle Errors Gracefully

    • Implement retry logic for failed submissions
    • Log rejected events for debugging
    • Monitor submission success rates

  4. Timestamp Accuracy

    • Use ISO 8601 format with timezone
    • Capture client-side occurrence time
    • Account for network latency in analysis

Monitoring

Track these metrics:

  • Event submission success rate
  • Event acceptance rate (post-validation)
  • Average submission latency
  • SIVT filtration rate

Frequently Asked Questions

  1. How quickly must events be reported?

    • Events should be reported within minutes of occurrence. Late-arriving events accepted up to 7 days for attribution purposes.

  2. What happens to events during API downtime?

    • Implement client-side queuing and retry logic. Events can be submitted retroactively once service restored.

  3. Can events be edited or deleted after submission?

    • No. Events are immutable once accepted. Submit correction events if needed.

  4. How are viewability standards applied to impression events?

    • Marketplaces should only report impressions meeting IAB viewability criteria (50% pixels, 1+ second). Topsort assumes reported impressions are viewable.

  5. What user identifier should be used?

    • Use a stable, pseudonymous hash of user ID. Must be consistent across sessions for same user. Do not send PII.