PDF security blog

PDF Tamper Detection in Go: net/http Client, errgroup Worker Pool and httptest

HTPBE Team··26 min read
PDF Tamper Detection in Go: net/http Client, errgroup Worker Pool and httptest

This article is a snapshot – content was accurate as of July 2026 (code examples tested against the API as of October 2026). The product evolves actively; specific counts, examples, and detection rules may have changed since publication – see the changelog for the current state.

A Go service that accepts PDFs from users has one job before it trusts them: find out whether the file was edited after it was created. The forged bank statement, the altered invoice and the doctored payslip all arrive through the same upload handler, and by the time a row is written to the database their claims are already part of your business logic. Ingress is the cheapest place to check.

This guide builds a small Go package around the PDF tamper detection API: a net/http client with per-attempt context timeouts, a retry policy that does not double-bill within a call, one typed error per documented error code, routing for each of the three verdicts, a worker pool that cancels itself when credits run out or the key is wrong, and tests that run entirely on test keys. The client is standard-library only; Step 9 adds errgroup, rate and singleflight from the golang.org/x modules. It does not repeat what the sibling guides cover. For the language-agnostic flow start with How to Detect PDF Tampering Programmatically; for other stacks see the Node.js, Python and Laravel / PHP guides. For queue-based throughput see batch verification with a job queue. What is specific to Go here is how errors, contexts, goroutines and httptest fit the API contract.

TL;DR

  • Two calls: POST /analyze returns {"id": ...}, GET /result/{id} returns the flat result with status of intact, modified or inconclusive.
  • POST /analyze spends a credit. Send an Idempotency-Key header and reuse the same value on every retry of that request, so a timeout never turns into a second charge.
  • Retry only what can succeed later: 429, 5xx, transport errors, and 409 idempotency_key_in_progress. Everything else is permanent.
  • Do not auto-reject on modified. Every result carries usage_caution.safe_for_automated_adverse_decision: false; route to a human.
  • Tests run on htpbe_test_* keys against the documented mock URLs, plus an httptest.Server for retry logic.
  • This is structural PDF tamper detection. It is not KYC, not OCR and not AI-text detection.

Prerequisites

  • Go 1.22 or newer (the handler below uses method-qualified ServeMux patterns; log/slog needs 1.21, math/rand/v2 needs 1.22)
  • An HTPBE? API key from the dashboard; the same page shows your test key
  • No third-party modules for the client itself (the optional storage snippet in Step 6 uses aws-sdk-go-v2)

Step 1: Try the Flow with curl and a Test Key

Test keys (prefix htpbe_test_) accept only the mock URLs under https://api.htpbe.tech/v1/test/, return deterministic results and consume no credits.

curl -X POST https://api.htpbe.tech/v1/analyze \
  -H "Authorization: Bearer $HTPBE_TEST_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://api.htpbe.tech/v1/test/clean.pdf"}'
# {"id":"00000000-0000-4000-8000-000000000001"}

curl https://api.htpbe.tech/v1/result/00000000-0000-4000-8000-000000000001 \
  -H "Authorization: Bearer $HTPBE_TEST_API_KEY"

POST /analyze is synchronous: when it returns 201 with the id, the result is already available. A live key accepts any public PDF URL up to 10 MB; the URL must be downloadable without authentication.

Step 2: The Result Type

The struct mirrors the documented GET /result/{id} response. encoding/json ignores unknown fields, so new fields never break decoding. Fields the API documents as nullable are pointers, so ‘absent’ stays distinguishable from zero.

package htpbe

const (
	StatusIntact       = "intact"
	StatusModified     = "modified"
	StatusInconclusive = "inconclusive"
)

// Result mirrors the GET /result/{id} response.
type Result struct {
	ID       string `json:"id"`
	Filename string `json:"filename"`
	FileSize int64  `json:"file_size"`
	// Unix seconds; null for test-key results and legacy records.
	CheckDate *int64 `json:"check_date"`

	AlgorithmVersion        string `json:"algorithm_version"`
	CurrentAlgorithmVersion string `json:"current_algorithm_version"`
	// Present only when the check ran on an older algorithm version.
	OutdatedWarning string `json:"outdated_warning"`

	// "intact" | "modified" | "inconclusive"
	Status string `json:"status"`
	// Present only when Status is "inconclusive":
	// consumer_software_origin | online_editor_origin | scanned_document |
	// unverifiable_metadata | filled_form_origin | fill_sign_origin
	StatusReason string `json:"status_reason"`

	// "certain" | "high" | "none"; null on legacy records.
	ModificationConfidence *string `json:"modification_confidence"`

	CreationDate     *int64  `json:"creation_date"`
	ModificationDate *int64  `json:"modification_date"`
	Creator          *string `json:"creator"`
	Producer         *string `json:"producer"`
	PDFVersion       *string `json:"pdf_version"`

	DateSequenceValid         bool `json:"date_sequence_valid"`
	MetadataCompletenessScore int  `json:"metadata_completeness_score"`

	XrefCount             int  `json:"xref_count"`
	HasIncrementalUpdates bool `json:"has_incremental_updates"`
	UpdateChainLength     int  `json:"update_chain_length"`

	HasDigitalSignature         bool `json:"has_digital_signature"`
	SignatureCount              int  `json:"signature_count"`
	SignatureRemoved            bool `json:"signature_removed"`
	ModificationsAfterSignature bool `json:"modifications_after_signature"`

	PageCount        int  `json:"page_count"`
	ObjectCount      int  `json:"object_count"`
	HasJavascript    bool `json:"has_javascript"`
	HasEmbeddedFiles bool `json:"has_embedded_files"`

	// Stable HTPBE_* ids, strongest first; empty unless Status is "modified".
	ModificationMarkers []string `json:"modification_markers"`

	UsageCaution UsageCaution `json:"usage_caution"`
}

// UsageCaution is the responsible-use guardrail present on every result.
type UsageCaution struct {
	SafeForAutomatedAdverseDecision bool `json:"safe_for_automated_adverse_decision"` // always false
	// "route_to_human_review" | "request_from_issuer" | "no_action"
	RecommendedAction string `json:"recommended_action"`
	Message           string `json:"message"`
}

Three fields deserve a closer look:

  • ModificationMarkers holds stable ids prefixed HTPBE_, for example HTPBE_SIGNATURE_REMOVED or HTPBE_DATES_DISAGREE. They never change once shipped, so branch on the id and render the label from the dictionary on the how-it-works page. Never parse labels.
  • ModificationConfidence is certain or high whenever Status is modified. Use it to prioritise the review queue, not to decide whether a human looks.
  • UsageCaution is part of the payload on purpose: a structural verdict describes the file, not the person who sent it.

Step 3: One Typed Error per Documented Code

Every non-2xx response has the shape {"error": "...", "code": "...", "details": "..."}. Wrap it in one type that carries the HTTP status, the machine-readable code and the Retry-After hint, so callers branch with errors.As instead of string matching.

package htpbe

import (
	"fmt"
	"time"
)

// Error codes documented for the API.
const (
	CodeInvalidRequest        = "invalid_request"
	CodeInvalidURLFormat      = "invalid_url_format"
	CodeDownloadFailed        = "download_failed"
	CodeMissingAPIKey         = "missing_api_key"
	CodeInvalidAPIKey         = "invalid_api_key"
	CodePaymentRequired       = "payment_required"
	CodeInactiveClient        = "inactive_client"
	CodeTestURLRequired       = "test_url_required"
	CodeAccountRestricted     = "account_restricted"
	CodeIdempotencyInProgress = "idempotency_key_in_progress"
	CodeFileTooLarge          = "file_too_large"
	CodeInvalidPDF            = "invalid_pdf"
	CodeIdempotencyMismatch   = "idempotency_key_fingerprint_mismatch"
	CodeServerAtCapacity      = "server_at_capacity"
	CodeInternalError         = "internal_error"
	CodeNotFound              = "not_found"
)

// APIError is returned for every non-2xx response.
type APIError struct {
	StatusCode int
	Code       string
	Message    string
	Details    string
	RetryAfter time.Duration // from the Retry-After header; 0 when absent
}

func (e *APIError) Error() string {
	return fmt.Sprintf("htpbe: HTTP %d %s: %s", e.StatusCode, e.Code, e.Message)
}

// Retryable reports whether sending the same request again can succeed.
func (e *APIError) Retryable() bool {
	switch {
	case e.StatusCode == 429, e.StatusCode >= 500:
		return true
	case e.StatusCode == 409 && e.Code == CodeIdempotencyInProgress:
		return true
	}
	return false
}

What each code means for your service:

StatuscodeMeaningWhat to do
400invalid_request, invalid_url_formatMalformed JSON, missing url, a non-HTTP(S) URL, or an oversized Idempotency-KeyBug in your code or input. Fix it; do not retry
400download_failedThe API could not fetch your URL (404, 403, auth required, 30 s download timeout, DNS)Check that the URL is public and live. The cause is on the URL side, so do not retry blindly
401missing_api_key, invalid_api_keyKey absent or not recognisedConfiguration error; alert, do not retry
402payment_requiredNo credit source left for this requestBilling event: top up on the pricing page; never retry
403inactive_client, account_restrictedKey deactivated or account restrictedContact support; do not retry
403test_url_requiredA test key was used with a real URLYou shipped a test key to production, or a real URL to a test run
409idempotency_key_in_progressA request with this Idempotency-Key is still being processedWait, retry with the same key
413file_too_largeThe PDF is over 10 MBReject the upload to the user
422invalid_pdfNot a PDF, corrupted, password-protected, or analysis could not completeAsk for a fresh, unprotected copy; retrying the same file is not expected to help
422idempotency_key_fingerprint_mismatchThe same key was sent with a different urlUse one key per document
429server_at_capacityServer-wide concurrency limit, not a per-key rate limitWait for Retry-After, retry
5xxinternal_errorTransient server errorRetry with backoff

GET /result/{id} adds 400 invalid_request for a malformed id and 404 not_found. The 404 also covers an id that belongs to another client, and a test key asking for anything other than the synthetic test ids.

Step 4: The Client

One Client is safe for concurrent use. The design decisions that matter:

  • Per-attempt timeouts, caller-owned deadline. Each HTTP attempt gets its own context.WithTimeout; the context you pass to Verify bounds the whole operation. The documented worst case for one analyze call is a 30 second download plus a 20 second analysis, so the default attempt budget is 55 seconds.
  • The POST is the billable step, so it carries an Idempotency-Key. A client-side timeout does not tell you whether the server finished. With the header, a retry of a request that already completed returns 201 with the same check id and spends no second credit. Without it, a blind retry can bill you twice.
  • The key is generated once per Verify call and reused on every attempt. You can pass your own to make retries safe across process restarts (see the caveat after the code).
  • Transport errors are retried, context cancellation is not.
package htpbe

import (
	"bytes"
	"context"
	"crypto/rand"
	"encoding/json"
	"errors"
	"fmt"
	"io"
	mrand "math/rand/v2"
	"net/http"
	"net/url"
	"strconv"
	"strings"
	"time"
)

const defaultBaseURL = "https://api.htpbe.tech/v1"

type Client struct {
	apiKey         string
	baseURL        string
	hc             *http.Client
	maxRetries     int
	attemptTimeout time.Duration
}

type Option func(*Client)

func WithBaseURL(u string) Option {
	return func(c *Client) { c.baseURL = strings.TrimRight(u, "/") }
}

func WithMaxRetries(n int) Option {
	return func(c *Client) { c.maxRetries = n }
}

func WithAttemptTimeout(d time.Duration) Option {
	return func(c *Client) { c.attemptTimeout = d }
}

func WithHTTPClient(hc *http.Client) Option {
	return func(c *Client) {
		if hc != nil {
			c.hc = hc
		}
	}
}

func New(apiKey string, opts ...Option) (*Client, error) {
	if apiKey == "" {
		return nil, errors.New("htpbe: API key is required")
	}
	c := &Client{
		apiKey:         apiKey,
		baseURL:        defaultBaseURL,
		hc:             &http.Client{}, // timeouts come from contexts, not http.Client.Timeout
		maxRetries:     3,
		attemptTimeout: 55 * time.Second,
	}
	for _, opt := range opts {
		opt(c)
	}
	return c, nil
}

type VerifyOptions struct {
	// OriginalFilename is stored and returned as Result.Filename.
	OriginalFilename string
	// IdempotencyKey identifies this logical request. Empty means "generate one
	// for this call". At most 255 characters; one key per document.
	IdempotencyKey string
}

// Verify submits pdfURL and returns the full result.
func (c *Client) Verify(ctx context.Context, pdfURL string, opts VerifyOptions) (*Result, error) {
	if pdfURL == "" {
		return nil, errors.New("htpbe: pdfURL is required")
	}
	key := opts.IdempotencyKey
	if key == "" {
		var err error
		if key, err = newIdempotencyKey(); err != nil {
			return nil, err
		}
	}

	body, err := json.Marshal(struct {
		URL              string `json:"url"`
		OriginalFilename string `json:"original_filename,omitempty"`
	}{pdfURL, opts.OriginalFilename})
	if err != nil {
		return nil, err
	}

	var created struct {
		ID string `json:"id"`
	}
	err = c.do(ctx, func(ctx context.Context) (*http.Request, error) {
		req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.baseURL+"/analyze", bytes.NewReader(body))
		if err != nil {
			return nil, err
		}
		req.Header.Set("Content-Type", "application/json")
		req.Header.Set("Idempotency-Key", key)
		return req, nil
	}, &created)
	if err != nil {
		return nil, err
	}
	if created.ID == "" {
		return nil, errors.New("htpbe: analyze response has no id")
	}

	var res Result
	err = c.do(ctx, func(ctx context.Context) (*http.Request, error) {
		return http.NewRequestWithContext(ctx, http.MethodGet, c.baseURL+"/result/"+url.PathEscape(created.ID), nil)
	}, &res)
	if err != nil {
		return nil, err
	}
	return &res, nil
}

// do runs one logical request: build, send, decode, retry when it makes sense.
func (c *Client) do(ctx context.Context, build func(context.Context) (*http.Request, error), out any) error {
	for attempt := 0; ; attempt++ {
		err := c.once(ctx, build, out)
		if err == nil {
			return nil
		}
		if ctx.Err() != nil || !isRetryable(err) || attempt >= c.maxRetries {
			return err
		}
		timer := time.NewTimer(backoff(attempt, err))
		select {
		case <-ctx.Done():
			timer.Stop()
			return ctx.Err()
		case <-timer.C:
		}
	}
}

func (c *Client) once(ctx context.Context, build func(context.Context) (*http.Request, error), out any) error {
	ctx, cancel := context.WithTimeout(ctx, c.attemptTimeout)
	defer cancel()

	req, err := build(ctx)
	if err != nil {
		return err
	}
	req.Header.Set("Authorization", "Bearer "+c.apiKey)

	resp, err := c.hc.Do(req)
	if err != nil {
		return fmt.Errorf("htpbe: %s %s: %w", req.Method, req.URL.Path, err)
	}
	defer resp.Body.Close()

	if resp.StatusCode/100 != 2 {
		return parseError(resp)
	}
	return json.NewDecoder(resp.Body).Decode(out)
}

// isRetryable: API errors follow Retryable(); anything else (connection reset,
// attempt timeout, truncated body) is a transport failure. Retrying is safe
// because the POST carries the same Idempotency-Key on every attempt.
func isRetryable(err error) bool {
	var apiErr *APIError
	if errors.As(err, &apiErr) {
		return apiErr.Retryable()
	}
	return true
}

func backoff(attempt int, err error) time.Duration {
	var apiErr *APIError
	if errors.As(err, &apiErr) && apiErr.RetryAfter > 0 {
		return apiErr.RetryAfter
	}
	d := min(time.Second<<attempt, 30*time.Second)
	return d + mrand.N(time.Second) // jitter
}

func parseError(resp *http.Response) error {
	raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<16))
	var eb struct {
		Error   string `json:"error"`
		Code    string `json:"code"`
		Details string `json:"details"`
	}
	_ = json.Unmarshal(raw, &eb) // the body may not be JSON (proxy error pages)

	e := &APIError{StatusCode: resp.StatusCode, Code: eb.Code, Message: eb.Error, Details: eb.Details}
	if e.Message == "" {
		e.Message = http.StatusText(resp.StatusCode)
	}
	if s, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && s > 0 {
		e.RetryAfter = min(time.Duration(s)*time.Second, 5*time.Minute)
	}
	return e
}

// newIdempotencyKey returns a random UUID v4.
func newIdempotencyKey() (string, error) {
	var b [16]byte
	if _, err := rand.Read(b[:]); err != nil {
		return "", err
	}
	b[6] = b[6]&0x0f | 0x40
	b[8] = b[8]&0x3f | 0x80
	return fmt.Sprintf("%x-%x-%x-%x-%x", b[0:4], b[4:6], b[6:8], b[8:10], b[10:]), nil
}

A caveat on caller-supplied keys: the server fingerprints the key against the url. If you reuse a key with a different URL you get 422 idempotency_key_fingerprint_mismatch. That matters with presigned URLs (Step 6): minting a fresh presigned URL for the same upload changes the url. When you retry across process restarts, persist the key together with the exact URL you sent, and replay both.

Step 5: Handle Each Verdict

The client returns facts; policy is yours. Two rules come straight from the contract. First, usage_caution.safe_for_automated_adverse_decision is always false, so no verdict is a licence to reject a person automatically. Second, inconclusive is neither a failure nor a pass: the integrity check does not apply to that kind of file.

package htpbe

// Action is where a document goes next.
type Action string

const (
	ActionAccept            Action = "accept"              // continue the automated path
	ActionHumanReview       Action = "human_review"        // a person looks before any decision
	ActionRequestFromIssuer Action = "request_from_issuer" // confirm with the issuing organisation
)

// Route maps a verdict to an action. An unknown status (for example one added
// to the API after you shipped) fails closed to human review.
func Route(r *Result) Action {
	switch r.Status {
	case StatusIntact:
		return ActionAccept
	case StatusModified:
		return ActionHumanReview
	case StatusInconclusive:
		return ActionRequestFromIssuer
	default:
		return ActionHumanReview
	}
}

// ReviewPriority orders the human queue; it never decides whether a human looks.
func ReviewPriority(r *Result) int {
	if r.ModificationConfidence != nil && *r.ModificationConfidence == "certain" {
		return 0 // first in line
	}
	return 1
}

How to treat each branch:

  • intact means no structural evidence of modification. It is not a guarantee of authenticity, so keep your own downstream checks (amounts, account lookups, sender authentication). Log OutdatedWarning if it is non-empty: the check ran on an older algorithm version and a re-analysis may give a different answer.
  • modified carries at least one marker in ModificationMarkers. Store the list with the case so the reviewer sees why. Consider ModificationsAfterSignature and SignatureRemoved as shortcuts for the signature-specific cases.
  • inconclusive has a StatusReason. Log it as a metric dimension, but route every reason the same way: the test documentation advises giving all inconclusive reasons one fallback path rather than branching on the exact string. For a document that claims institutional origin (a bank statement, a payslip), consumer_software_origin is itself worth a second look, since a real bank or payroll system does not distribute files that way. What ‘inconclusive’ means goes deeper.

Mapping errors in an HTTP handler

A handler also has to translate *APIError into something sensible for its own caller, without leaking billing or key problems.

package main

import (
	"context"
	"encoding/json"
	"errors"
	"log/slog"
	"net/http"
	"os"
	"time"

	"yourapp/htpbe"
)

type verifyRequest struct {
	DocumentURL      string `json:"document_url"`
	OriginalFilename string `json:"original_filename"`
}

func main() {
	client, err := htpbe.New(os.Getenv("HTPBE_API_KEY"))
	if err != nil {
		slog.Error("htpbe init failed", "err", err)
		os.Exit(1)
	}

	mux := http.NewServeMux()
	mux.HandleFunc("POST /documents", func(w http.ResponseWriter, r *http.Request) {
		var in verifyRequest
		if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<16)).Decode(&in); err != nil || in.DocumentURL == "" {
			writeJSON(w, http.StatusBadRequest, map[string]string{"error": "document_url is required"})
			return
		}

		ctx, cancel := context.WithTimeout(r.Context(), 90*time.Second)
		defer cancel()

		res, err := client.Verify(ctx, in.DocumentURL, htpbe.VerifyOptions{OriginalFilename: in.OriginalFilename})
		if err != nil {
			status, msg := statusFor(err)
			writeJSON(w, status, map[string]string{"error": msg})
			return
		}

		switch htpbe.Route(res) {
		case htpbe.ActionAccept:
			writeJSON(w, http.StatusOK, map[string]any{"decision": "accept", "check_id": res.ID})
		case htpbe.ActionRequestFromIssuer:
			slog.Info("inconclusive", "check_id", res.ID, "reason", res.StatusReason)
			writeJSON(w, http.StatusAccepted, map[string]any{"decision": "request_from_issuer", "check_id": res.ID})
		default:
			slog.Info("modified", "check_id", res.ID, "markers", res.ModificationMarkers)
			writeJSON(w, http.StatusAccepted, map[string]any{"decision": "human_review", "check_id": res.ID})
		}
	})

	srv := &http.Server{Addr: ":8080", Handler: mux, ReadHeaderTimeout: 5 * time.Second}
	slog.Error("server stopped", "err", srv.ListenAndServe())
}

// statusFor translates a client error into our own response. Key, billing and
// test-key problems are our configuration, so the caller only sees 503.
func statusFor(err error) (int, string) {
	var apiErr *htpbe.APIError
	if !errors.As(err, &apiErr) {
		slog.Error("htpbe verify failed", "err", err)
		return http.StatusBadGateway, "verification failed"
	}
	switch apiErr.Code {
	case htpbe.CodeMissingAPIKey, htpbe.CodeInvalidAPIKey, htpbe.CodePaymentRequired,
		htpbe.CodeInactiveClient, htpbe.CodeAccountRestricted, htpbe.CodeTestURLRequired:
		slog.Error("htpbe unavailable: needs operator attention", "code", apiErr.Code)
		return http.StatusServiceUnavailable, "verification temporarily unavailable"
	case htpbe.CodeInvalidURLFormat, htpbe.CodeDownloadFailed:
		return http.StatusUnprocessableEntity, "the document could not be downloaded"
	case htpbe.CodeFileTooLarge:
		return http.StatusRequestEntityTooLarge, "PDF must be under 10 MB"
	case htpbe.CodeInvalidPDF:
		return http.StatusUnprocessableEntity, "the file could not be read as a PDF"
	}
	slog.Error("htpbe verify failed", "code", apiErr.Code, "status", apiErr.StatusCode)
	return http.StatusBadGateway, "verification failed"
}

func writeJSON(w http.ResponseWriter, status int, v any) {
	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(status)
	_ = json.NewEncoder(w).Encode(v)
}

Verification inside the request handler blocks for the analysis time (typically 2 to 5 seconds, longer for large or complex files). That suits low-volume onboarding; for high-volume intake, return 202 immediately and verify from a worker, as in the queue guide.

Step 6: Giving the API a Reachable URL

The API downloads the file itself; it does not accept uploads. Store the upload privately, mint a short-lived presigned GET URL and pass that. With an S3-compatible store the shape is:

presigned, err := presigner.PresignGetObject(ctx, &s3.GetObjectInput{
	Bucket: &bucket, Key: &key,
}, s3.WithPresignExpires(5*time.Minute))
if err != nil {
	return nil, err
}
return client.Verify(ctx, presigned.URL, htpbe.VerifyOptions{OriginalFilename: name})

Pass OriginalFilename, otherwise Result.Filename is the opaque object key. The same pattern works for Google Cloud Storage signed URLs and Cloudflare R2. One safety note on your side: every Verify call spends a credit, so never pass a caller-supplied URL straight through. Accept only URLs your own service minted (such as the presigned URLs above) or hosts on an allowlist you control.

Step 7: Bounded Concurrency for a Backlog

For a backlog, run a fixed pool of workers over one shared Client (it reuses connections). Indexing the output slice by job position keeps results ordered without a mutex.

type Outcome struct {
	URL    string
	Result *htpbe.Result
	Err    error
}

func VerifyAll(ctx context.Context, c *htpbe.Client, urls []string, workers int) []Outcome {
	workers = max(workers, 1)
	out := make([]Outcome, len(urls))
	jobs := make(chan int)

	var wg sync.WaitGroup
	for range workers {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for i := range jobs {
				res, err := c.Verify(ctx, urls[i], htpbe.VerifyOptions{})
				out[i] = Outcome{URL: urls[i], Result: res, Err: err}
			}
		}()
	}
	for i := range urls {
		jobs <- i
	}
	close(jobs)
	wg.Wait()
	return out
}

429 server_at_capacity is server-wide, not per key, so a small worker count is the first defence and the Retry-After-aware retry in the client is the second. Check each Outcome.Err with errors.As: a payment_required mid-batch means the rest will fail the same way, so cancel the context you passed in (for example from a context.WithCancel wrapper that fires on that code) rather than collecting hundreds of identical errors.

Step 8: Tests on Test Keys

Two layers, both without spending a credit.

Retry logic with httptest. No key or network is needed. The server fails the first POST, then succeeds; the test asserts the retry reused the same Idempotency-Key and that the GET ran exactly once.

package htpbe_test

import (
	"context"
	"errors"
	"net/http"
	"net/http/httptest"
	"sync/atomic"
	"testing"
	"time"

	"yourapp/htpbe"
)

func TestVerifyRetriesPOSTWithSameIdempotencyKey(t *testing.T) {
	var posts, gets atomic.Int32
	keys := make(chan string, 4)

	mux := http.NewServeMux()
	mux.HandleFunc("POST /analyze", func(w http.ResponseWriter, r *http.Request) {
		keys <- r.Header.Get("Idempotency-Key")
		if posts.Add(1) == 1 {
			w.Header().Set("Content-Type", "application/json")
			w.WriteHeader(http.StatusInternalServerError)
			_, _ = w.Write([]byte(`{"error":"Failed to analyze PDF","code":"internal_error"}`))
			return
		}
		w.Header().Set("Content-Type", "application/json")
		w.WriteHeader(http.StatusCreated)
		_, _ = w.Write([]byte(`{"id":"00000000-0000-4000-8000-000000000001"}`))
	})
	mux.HandleFunc("GET /result/{id}", func(w http.ResponseWriter, r *http.Request) {
		gets.Add(1)
		w.Header().Set("Content-Type", "application/json")
		_, _ = w.Write([]byte(`{"id":"` + r.PathValue("id") + `","status":"intact"}`))
	})
	srv := httptest.NewServer(mux)
	defer srv.Close()

	c, _ := htpbe.New("htpbe_test_dummy", htpbe.WithBaseURL(srv.URL), htpbe.WithMaxRetries(2))
	ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
	defer cancel()

	res, err := c.Verify(ctx, "https://api.htpbe.tech/v1/test/clean.pdf", htpbe.VerifyOptions{})
	if err != nil {
		t.Fatal(err)
	}
	if res.Status != htpbe.StatusIntact {
		t.Fatalf("status = %q", res.Status)
	}
	if posts.Load() != 2 || gets.Load() != 1 {
		t.Fatalf("posts=%d gets=%d, want 2 and 1", posts.Load(), gets.Load())
	}
	first, second := <-keys, <-keys
	if first == "" || first != second {
		t.Fatalf("Idempotency-Key changed between attempts: %q vs %q", first, second)
	}
}

func TestPermanentErrorsAreNotRetried(t *testing.T) {
	var posts atomic.Int32
	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		posts.Add(1)
		w.WriteHeader(http.StatusPaymentRequired)
		_, _ = w.Write([]byte(`{"error":"No credits remaining.","code":"payment_required"}`))
	}))
	defer srv.Close()

	c, _ := htpbe.New("htpbe_test_dummy", htpbe.WithBaseURL(srv.URL))
	_, err := c.Verify(context.Background(), "https://example.com/a.pdf", htpbe.VerifyOptions{})

	var apiErr *htpbe.APIError
	if !errors.As(err, &apiErr) || apiErr.Code != htpbe.CodePaymentRequired {
		t.Fatalf("err = %v", err)
	}
	if posts.Load() != 1 {
		t.Fatalf("posts = %d, want exactly 1", posts.Load())
	}
}

The real API on a test key. Add "os" to the imports of the same test file, and gate these on an environment variable so CI without a key skips them. Test keys only accept the documented mock URLs, return deterministic results, and never debit credits.

func TestLiveTestKey(t *testing.T) {
	key := os.Getenv("HTPBE_TEST_API_KEY")
	if key == "" {
		t.Skip("HTPBE_TEST_API_KEY not set")
	}
	c, _ := htpbe.New(key)
	const base = "https://api.htpbe.tech/v1/test/"

	cases := []struct {
		file   string
		status string
		action htpbe.Action
	}{
		{"clean.pdf", htpbe.StatusIntact, htpbe.ActionAccept},
		{"signature-removed.pdf", htpbe.StatusModified, htpbe.ActionHumanReview},
		{"dates-mismatch.pdf", htpbe.StatusModified, htpbe.ActionHumanReview},
		{"inconclusive.pdf", htpbe.StatusInconclusive, htpbe.ActionRequestFromIssuer},
		{"scanned-document.pdf", htpbe.StatusInconclusive, htpbe.ActionRequestFromIssuer},
	}
	for _, tc := range cases {
		t.Run(tc.file, func(t *testing.T) {
			ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
			defer cancel()
			res, err := c.Verify(ctx, base+tc.file, htpbe.VerifyOptions{})
			if err != nil {
				t.Fatal(err)
			}
			if res.Status != tc.status || htpbe.Route(res) != tc.action {
				t.Fatalf("status=%q action=%q", res.Status, htpbe.Route(res))
			}
		})
	}

	t.Run("out of credits", func(t *testing.T) {
		_, err := c.Verify(context.Background(), base+"trigger-402.pdf", htpbe.VerifyOptions{})
		var apiErr *htpbe.APIError
		if !errors.As(err, &apiErr) || apiErr.Code != htpbe.CodePaymentRequired {
			t.Fatalf("err = %v", err)
		}
	})
}

Notes that save debugging time:

  • trigger-402.pdf is the only way to exercise the 402 path with a test key, because a test key never debits credits. It has no check id and cannot be passed to GET /result/{id}.
  • Do not assert on CheckDate: it is null for synthetic results.
  • A test key resolves only the synthetic ids. Asking for any other id, even a real check from the same account, returns 404 not_found.
  • Every mock URL and its expected verdict are listed in the public API documentation. Keep test and live keys in separate environment files and never commit either.

Step 9: A Worker Pool That Stops When the Money Does

VerifyAll in Step 7 is fine for a script. A long-running service needs three more behaviours, and each has a Go-idiomatic home: a client-side pace, a pool that cancels itself on a configuration or billing failure, and a clean stop on SIGTERM so a deploy never abandons a request mid-flight. errgroup, rate.Limiter and singleflight cover all three without a queue library.

package main

import (
	"context"
	"errors"
	"log/slog"
	"os"
	"os/signal"
	"syscall"

	"golang.org/x/sync/errgroup"
	"golang.org/x/sync/singleflight"
	"golang.org/x/time/rate"

	"example.com/app/htpbe"
)

// Job is one stored document. URL is a presigned GET minted for this run.
type Job struct {
	Hash string // SHA-256 of the file bytes
	URL  string
}

type Checker struct {
	client  *htpbe.Client
	limiter *rate.Limiter
	flight  singleflight.Group
}

// Check collapses concurrent calls for the same content hash into one request,
// so a file that lands twice in the same minute is analysed, and charged, once.
func (k *Checker) Check(ctx context.Context, j Job) (*htpbe.Result, error) {
	v, err, _ := k.flight.Do(j.Hash, func() (any, error) {
		if err := k.limiter.Wait(ctx); err != nil {
			return nil, err
		}
		return k.client.Verify(ctx, j.URL, htpbe.VerifyOptions{})
	})
	if err != nil {
		return nil, err
	}
	return v.(*htpbe.Result), nil
}

// fatal reports errors after which every remaining job would fail the same way.
func fatal(err error) bool {
	var apiErr *htpbe.APIError
	if !errors.As(err, &apiErr) {
		return false
	}
	switch apiErr.Code {
	case htpbe.CodePaymentRequired, htpbe.CodeInvalidAPIKey,
		htpbe.CodeInactiveClient, htpbe.CodeAccountRestricted,
		htpbe.CodeTestURLRequired:
		return true
	}
	return false
}

func Run(ctx context.Context, k *Checker, jobs []Job, workers int) error {
	g, ctx := errgroup.WithContext(ctx)
	g.SetLimit(workers)

	for _, j := range jobs {
		if ctx.Err() != nil {
			break // a fatal error or SIGTERM: stop handing out work
		}
		g.Go(func() error {
			res, err := k.Check(ctx, j)
			switch {
			case err == nil:
				slog.Info("checked", "hash", j.Hash, "status", res.Status,
					"action", htpbe.Route(res))
				return nil
			case fatal(err):
				return err // errgroup cancels ctx for everyone
			case ctx.Err() != nil:
				return nil // shutting down; the job stays pending
			default:
				slog.Warn("check failed", "hash", j.Hash, "err", err)
				return nil // a per-document failure must not stop the batch
			}
		})
	}
	return g.Wait()
}

func main() {
	ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
	defer stop()

	client, err := htpbe.New(os.Getenv("HTPBE_API_KEY"))
	if err != nil {
		slog.Error("client", "err", err)
		os.Exit(1)
	}
	// 5 requests per second with a burst of 5 is a pace you choose, not an API limit.
	k := &Checker{client: client, limiter: rate.NewLimiter(5, 5)}

	if err := Run(ctx, k, loadPending(), 4); err != nil {
		slog.Error("batch stopped", "err", err)
		os.Exit(1)
	}
}

Three details are worth the lines they take:

  • Why fatal exists. 402 payment_required means no credit source is left, and 401, 403 and test_url_required mean configuration. Retrying or continuing is wasted effort and, for 402, noise in your logs. Returning the error from the errgroup goroutine cancels the shared context, so the in-flight workers unwind and the loop stops handing out work. Per-document errors (an unreachable URL, a file that is not a valid PDF) return nil so one bad file never stops the batch.
  • What cancellation does to a request in flight. Verify takes the context, so a cancelled context aborts the HTTP attempt. If the request had already reached the API, the Idempotency-Key the client generated for it is lost with the process. Treat a job interrupted that way as pending and run it again with a new key on the next start; the contract allows a re-submission to spend another credit, which is the price of not persisting keys. If you cannot accept that, store a key on the job row before the call and pass it as VerifyOptions.IdempotencyKey.
  • singleflight is per process. It shares the first caller’s context with the waiters, so cancel with care, and it does not deduplicate across replicas. Cross-instance dedup needs the hash in a database with a unique constraint, the pattern in the queue guide.

The pool is deliberately small. 429 server_at_capacity is server-wide, so four workers and a modest limiter beat sixty workers that spend their time in backoff.

Retry Policy at a Glance

SituationRetry?Same Idempotency-Key?
Connection error, attempt timeout, truncated bodyYes, with backoffYes
429 server_at_capacityYes, after Retry-AfterYes
409 idempotency_key_in_progressYes, after Retry-AfterYes, never a new key
5xxYes, with backoffYes
Parent context cancelled or past its deadlineNon/a
400, 401, 402, 403, 413, 422Non/a

The one rule to keep: a new key means a new, separately charged request. Never mint a fresh key just to get past a 409.

What This Does Not Catch

Structural analysis has honest limits, and a service that automates decisions should encode them:

  • Content that was false from the start. The check finds evidence that a file was changed after it was created. A document whose figures were wrong when it was first produced carries no such evidence, because nothing was changed afterwards. That is why a file produced by consumer software comes back inconclusive rather than intact.
  • Skilled, deliberate fabrication. A well-resourced forger can sometimes produce a file that shows no structural sign of editing. It’s rare and takes effort, but intact means no modification was detected, not that the document is genuine. Confirmation with the issuer remains the stronger check.
  • Password-protected PDFs. They come back as 422 invalid_pdf; the password has to be removed before submitting.
  • Cryptographic signature validity. The API reports whether a signature is present, was removed, or was followed by edits. It does not validate the certificate chain.

Pair the structural verdict with domain checks, such as amount validation, account lookups and sender authentication, and your KYC or OCR provider. See PDF Fraud Prevention Best Practices.

Before You Ship

  1. Set HTPBE_API_KEY from the environment and fail fast at startup when it is missing.
  2. Alert on payment_required, invalid_api_key and test_url_required: all three mean configuration, not user input.
  3. Decide where verification runs (inline for low volume, a worker for high volume) and give every call a deadline.
  4. Put human_review and request_from_issuer behind real queues with owners; the integration is only as good as what happens to those cases.

New accounts get five checks to try, then pay-per-check credits or a subscription (see current pricing). Copy your test key, run the curl call from Step 1, then the httptest suite above. The API reference documents every response field and error code the client branches on.

Share this article

Found this article helpful? Share it with others to spread knowledge about PDF security and fraud detection.

https://htpbe.tech/blog/pdf-verification-go-integration-guide

Secure your workflow

Create your account – check PDFs on the web or with an API key, both ready on signup.
No sales call. Cancel any time.