Enterprise security. Built for what’s next.
← All essays

Post-quantum security / Essay 03

Two identities. One secret. What post-quantum mTLS has to prove.

A quantum-resistant key exchange is only one part of a mutually authenticated connection. Certificates, signatures, trust policy, and lifecycle operations must make the journey too.

A detailed monochrome natural-history illustration of a pangolin partly curled into an armored spiral.
Overlapping scales provide the visual motif for an essay about layered cryptographic responsibilities. Original editorial illustration.
Reading size

The green padlock leaves a lot unsaid

An engineer inspects a connection and sees that it negotiated a post-quantum hybrid group. That is useful information. It is not the end of the inspection. Which certificate did the server present? Did the server request a client certificate? Which algorithm signed each proof? What trust policy did each side actually enforce?

Mutual TLS, or mTLS, is a relationship between two authenticated peers, not a synonym for “strong encryption.” In TLS 1.3, certificate-based authentication uses certificate messages and proofs tied to the handshake. Key establishment supplies the secret material from which traffic keys are derived. These mechanisms cooperate, but changing one does not automatically change the other.1

This distinction is easy to miss because a product dashboard often compresses a complex negotiation into a single status. A successful migration needs a more precise vocabulary. Transport confidentiality, server identity, client identity, and application authorization should each have an answer that can be inspected.

Rows of server racks and cabling in a data center.
A cryptographic migration crosses many termination points. Inventory the gateways, proxies, connectors, and application servers behind a connection.Photography: imgix / Unsplash

A KEM is not a certificate signature

NIST’s ML-KEM standard specifies a key-encapsulation mechanism for establishing a shared secret. ML-DSA is a digital-signature standard. One contributes to secret establishment; the other can support evidence that a holder of a private signing key approved particular data. Their names are similar because both are lattice-based standards, not because they do the same job.23

Hybrid key agreement combines a traditional mechanism with a post-quantum mechanism through a specified construction. RFC 10024 defines hybrid groups for TLS 1.3. The group name is a meaningful record of the key-agreement choice; it is not a declaration that every certificate in the chain uses a post-quantum signature.4

LayerQuestion to ask
Key establishmentWhich traditional and post-quantum contributions enter the negotiated construction?
Server authenticationWhich chain and handshake signature does the client verify?
Client authenticationIs a client certificate required, and what identity does it assert?
AuthorizationWhich resource and action may that authenticated identity access?

For an enterprise, that final row matters as much as the first. A certificate can identify a device or workload without entitling it to an entire network. The service still needs to connect cryptographic identity to an application-level decision. Otherwise the migration can improve the mathematics while preserving an unnecessarily broad access model.

The new mathematics has a size

Algorithm choices affect how much material a handshake carries. NIST publishes concrete byte sizes for keys, ciphertexts, and signatures. These are useful planning inputs because they are defined by the parameter sets. They are not measurements of a complete TLS exchange.23

Post-quantum security has a byte budgetStandardized public-key and output sizes, in bytes
02,0004,000ML-KEM-512800768ML-KEM-7681,1841,088ML-KEM-10241,5681,568ML-DSA-441,3122,420ML-DSA-651,9523,309ML-DSA-872,5924,627
Ciphertext / signaturePublic key

Encoding sizes from FIPS 203, Table 3, and FIPS 204, Table 2. These are individual algorithm objects, not full TLS handshakes. Certificates, chain length, classical hybrid material, and framing add bytes.

View data & methodology

Source: NIST FIPS 203 and FIPS 204 (2024).

AlgorithmPublic key (bytes)Ciphertext or signature (bytes)Output type
ML-KEM-512800768Ciphertext
ML-KEM-76811841088Ciphertext
ML-KEM-102415681568Ciphertext
ML-DSA-4413122420Signature
ML-DSA-6519523309Signature
ML-DSA-8725924627Signature
Download CSV ↓

For ML-KEM-768, the encapsulation key and ciphertext together occupy 2,272 bytes. That sum excludes the traditional hybrid contribution, TLS framing, certificates, and other messages. ML-DSA-65 has a 3,309-byte signature, but a certificate contains more than a signature. A chain contains more than one certificate. Multiplying one number by two is not a reliable estimate of mutual authentication overhead.

The transport turns these sizes into operational questions. Will a larger initial flight fragment? How does loss affect completion? Does a constrained client verify the chain within an acceptable budget? An efficient algorithm can still meet a fragile middlebox or an application that assumes a smaller message. Test the path, not just a cryptographic microbenchmark.

Follow both proofs through the handshake

TLS 1.3 mutual authentication: a simplified message order
ClientServer1. ClientHello + supported groups and key share2. ServerHello + selected hybrid key share3. EncryptedExtensions + CertificateRequest4. Server Certificate + CertificateVerify + Finished5. Client Certificate + CertificateVerify + Finished6. Application data within authorized scope
Read the sequence as text
  1. ClientServer: ClientHello + supported groups and key share
  2. ServerClient: ServerHello + selected hybrid key share
  3. ServerClient: EncryptedExtensions + CertificateRequest
  4. ServerClient: Server Certificate + CertificateVerify + Finished
  5. ClientServer: Client Certificate + CertificateVerify + Finished
  6. ClientServer: Application data within authorized scope

The diagram emphasizes ordering, not every extension or resumption branch. A deployment must establish which signature schemes each peer can verify, which certificate authorities it trusts, and which identities it accepts. Mutual authentication requires the server to validate the client as well as the client validating the server.1

A migration test should deliberately present the wrong client identity, an untrusted issuer, an expired credential, and a revoked enrollment. It should also test a peer that understands the key-agreement group but cannot process the intended authentication scheme. A clean rejection is often more informative than another successful connection.

The migration starts before the handshake

Cryptographic inventory is not merely a list of libraries. It is a map of where cryptography is used and which systems depend on it. NIST’s migration work emphasizes identifying those dependencies so organizations can prioritize and plan change.5

For mTLS, inventory the enrollment service, certificate authority, issuing profiles, trust distribution, client libraries, gateways, proxies, hardware-backed key storage, and renewal jobs. Identify which team owns each failure. A tunnel team may be able to enable a hybrid group quickly while a separate PKI team needs a longer schedule for certificate support.

A practical migration sequence
  1. 01Discover

    Find protocols, peers, issuers, and owners.

  2. 02Prove

    Exercise negotiation and negative trust cases.

  3. 03Pilot

    Roll through representative clients and paths.

  4. 04Operate

    Renew, revoke, observe, and recover.

Keep the rollback policy explicit. Compatibility can justify a staged transition, but a silent downgrade can erase the very assurance the organization intended to add. Record what was negotiated and why an exception exists. Give exceptions an owner and an expiry. A successful fallback should not be indistinguishable from the intended security posture.

What this means for Antara

Antara connects post-quantum transport, client and server certificate exchange, and mTLS to enterprise access policy. The architecture conversation should therefore include both peers and the PKI that supplies their identities. Our certificate-exchange lab makes the distinction tangible: change a trust condition and observe whether the connection remains eligible.

An evaluation should collect the negotiated group, peer certificate details, validated identity, policy result, and reason for failure. It should then repeat the exercise during renewal and revocation. These are not housekeeping tasks after the cryptography is finished. They are how the cryptographic relationship continues to mean something over time.

The goal is a connection whose properties can be explained without relying on a padlock icon. Two peers establish a secret. Each proves an identity under a defined trust policy. The resulting session receives a bounded permission to do work. Post-quantum migration is strongest when all three statements remain true together.

Sources & further reading

Primary specifications and original technical documentation underpin the explanations and data in this essay.

  1. RFC 9846: TLS 1.3
  2. NIST FIPS 203: ML-KEM, Table 3
  3. NIST FIPS 204: ML-DSA, Table 2
  4. RFC 10024: hybrid key agreement for TLS 1.3
  5. NIST NCCoE: migration to post-quantum cryptography