Yin YangPutzi San

Common Cryptographic File Formats

Below is an overview of common file formats and standards for storing cryptographic keys and certificates. This includes examples of generating, importing, and exporting keys in various formats using OpenSSL and the Web Crypto API.

PEM (Privacy-Enhanced Mail)

  • Description: A text-based, Base64-encoded format enclosed between headers like -----BEGIN ...----- and -----END ...-----.
  • Typical Use: Commonly used to store keys, certificates, and related data in a human-readable form; frequently used on Unix-like systems.
  • Extensions: .pem, .crt, .cer, .key, .pub, .csr (despite the extension, .csr files are usually PEM-encoded)
  • Formal Specification: RFC 7468
    • In context of asymmetric keys, the encoded data should be a DER encoded ASN.1 PrivateKeyInfo structure as described in PKCS #8

Example:

-----BEGIN PUBLIC KEY-----
MIIBIjANBgkq...
-----END PUBLIC KEY-----

Generate a private key in PEM format using OpenSSL:

openssl genpkey -algorithm RSA -out private_key.pem

Import a private key in PEM format using the Web Crypto API:

/**
 * @param {string} pem
 * @return {Promise<CryptoKey>}
 */
function importPrivateKey(pem) {
  const base64 = pem
    .replace(/-----BEGIN PRIVATE KEY-----/, "")
    .replace(/-----END PRIVATE KEY-----/, "")
    .replace(/\s/g, ""); // Remove all whitespace characters
  const keyData = new Uint8Array(
    atob(base64)
      .split("")
      .map((c) => c.charCodeAt(0)),
  );
  return crypto.subtle.importKey("pkcs8", keyData, { name: "RSASSA-PKCS1-v1_5", hash: "SHA-256" }, true, ["decrypt"]);
}

Export a public key in PEM format using the Web Crypto API:

/**
 * @param {CryptoKey} key
 * @return {string}
 */
async function exportPublicKeyToPem(key) {
  // for private key use "pkcs8" instead of "spki"
  const publicKeyData = await crypto.subtle.exportKey("spki", key);
  const base64 = btoa(String.fromCharCode(...new Uint8Array(publicKeyData)));
  const pemFormattedBlocks = base64.match(/.{1,64}/g)?.join("\n") || ""; // Format the Base64 string into PEM blocks
  return `-----BEGIN PUBLIC KEY-----\n${pemFormattedBlocks}\n-----END PUBLIC KEY-----`;
}

DER (Distinguished Encoding Rules)

  • Description: A binary encoding of ASN.1 structures.
  • Typical Use: Used where a compact, non-text format is preferred, sometimes required by certain software or devices.
  • Extensions: .der, .cer
  • Formal Specification: ITU-T X.690
  • (No readable example due to binary nature.)

Generate a private key in DER format using OpenSSL:

openssl genpkey -algorithm RSA -outform DER -out private_key.der

Import a private key in DER format using the Web Crypto API:

/**
 * @param {ArrayBuffer} der
 * @return {Promise<CryptoKey>}
 */
function importPrivateKey(der) {
  return crypto.subtle.importKey("pkcs8", der, { name: "RSASSA-PKCS1-v1_5", hash: "SHA-256" }, true, ["decrypt"]);
}

Export a public key in DER format using the Web Crypto API:

/**
 * @param {CryptoKey} key
 * @return {Promise<ArrayBuffer>}
 */
async function exportPublicKeyToDer(key) {
  return crypto.subtle.exportKey("spki", key);
}

JWK (JSON Web Key)

  • Description: A JSON-based format for representing keys, primarily used in web ecosystems.
  • Key Fields:
    • kty: Key type (e.g., RSA, EC, oct for symmetric keys).
    • k: For symmetric keys, this contains the actual key material in base64url-encoded form. For asymmetric keys, other parameters are used (n for modulus in RSA, e for exponent, etc.).
    • Other fields like alg, use, kid provide hints about the key’s intended algorithm, usage (signature or encryption), and an identifier. See RFC 7517 for all parameters.
  • Typical Use: Popular in modern web-based security protocols (OAuth 2.0, OpenID Connect) and for JSON-based token handling.
  • Extensions: .json, .jwk
  • Formal Specification: RFC 7517 (JSON Web Key) and RFC 7518 (JSON Web Algorithms - definitions on how e.g. an RSA key is represented in JWK)

The JWK format explicitly is not an encoding standard for ASN.1 structures like DER. Instead, it has an own RFC (RFC 7518) that defines the key values for different key types and algorithms.

Example (Symmetric Key JWK):

{
  "kty": "oct",
  "k": "hXN9tJhrU7qSViY4YsIzKg"
}

Openssl does not directly support JWK format.

Import a private key in JWK format using the Web Crypto API:

/**
 * @param {object} jwk
 * @return {Promise<CryptoKey>}
 */
function importPrivateKey(jwk) {
  return crypto.subtle.importKey("jwk", jwk, { name: "RSASSA-PKCS1-v1_5", hash: "SHA-256" }, true, ["decrypt"]);
}

Export a public key in JWK format using the Web Crypto API:

/**
 * @param {CryptoKey} key
 * @return {Promise<object>}
 */
async function exportPublicKeyToJwk(key) {
  return crypto.subtle.exportKey("jwk", key);
}

JWKS (JSON Web Key Set)

  • Description: A JSON array of JWKs for serving multiple keys.
  • Typical Use: Used by servers to publish public keys for verification, allowing clients to fetch current keys from a single endpoint.
  • Extensions: .json, .jwks

Example:

{
  "keys": [
    {
      "kty": "RSA",
      "n": "0vx7agoebGcQ...",
      "e": "AQAB",
      "alg": "RS256",
      "use": "sig"
    }
  ]
}

In Depth: Cryptographic File Formats and Standards

Understanding ASN.1, Standards (e.g., PKCS#8), and Encoding Formats (e.g., PEM)

When dealing with cryptographic data structures, it's essential to distinguish between ASN.1, standards like PKCS#8, and encoding formats like PEM. Here’s a concise breakdown:

  • ASN.1: A framework for defining data structures. It is abstract and not tied to any specific encoding format. For instance, ASN.1 defines the fields a private key contains but does not specify how it should be stored or transmitted.
  • Standards (e.g., PKCS#8): These utilize ASN.1 to define the structure of specific data types, such as private keys. PKCS#8 specifies the fields and their arrangement but leaves encoding and transmission details to other layers.
  • Encoding Formats (e.g., PEM): These provide a concrete representation of ASN.1-structured data for storage or transmission. For example, PEM encodes ASN.1 structures, such as private keys defined by PKCS#8, into a format suitable for transport.
    • JWK is an exception as it is a JSON-based format and not an encoding of ASN.1 structures.

Tools like OpenSSL often operate with specific encoding formats, such as PEM, and include mechanisms to parse and translate these formats into internal ASN.1 structures for further processing.

ASN.1 and DER: How They Work Together

1. ASN.1 (Abstract Syntax Notation One)

ASN.1 is a language for describing data structures. It specifies the fields, types, and order of those fields. For example:

MyStructure ::= SEQUENCE {
    a INTEGER,
    b BOOLEAN,
    c OCTET STRING
}

Here, MyStructure has three fields: a, b, and c. ASN.1 only defines these fields at a conceptual level and does not indicate how they will be stored in bytes.

2. DER (Distinguished Encoding Rules)

DER is a set of rules for converting an ASN.1 structure into a series of bytes. It follows a Tag-Length-Value (TLV) pattern:

  • Tag: Indicates the data type (e.g., INTEGER, BOOLEAN).
  • Length: Specifies the size of the value in bytes.
  • Value: Holds the actual data.

DER requires a unique way to encode each data type. It ensures that a given ASN.1 structure can only be encoded one way.

3. Illustrative Example

Using the ASN.1 definition above, suppose:

  • a = 42
  • b = TRUE
  • c = "hello"

The DER encoding might be shown in hexadecimal as follows:

30 0A                  -- SEQUENCE (10 bytes total)
   02 01 2A            -- INTEGER (42)
   01 01 FF            -- BOOLEAN (TRUE)
   04 05 68 65 6C 6C 6F -- OCTET STRING ("hello")
  • 30 is the tag for SEQUENCE.
  • 0A is the length for everything inside the SEQUENCE.
  • 02 01 2A encodes an INTEGER with value 42.
  • 01 01 FF encodes a BOOLEAN with value TRUE.
  • 04 05 68 65 6C 6C 6F encodes an OCTET STRING holding the bytes h e l l o.

4. Linking the Schema and the Encoding

In this example, DER does not include the names a, b, or c. It only encodes the types and values in the order defined by the ASN.1 schema. A decoder uses the same schema to know that:

  1. The first field in the SEQUENCE is a (INTEGER).
  2. The second field is b (BOOLEAN).
  3. The third field is c (OCTET STRING).

If the decoder did not have the ASN.1 schema, it would still see an INTEGER, a BOOLEAN, and an OCTET STRING, but it would not know these fields are called a, b, and c.

Summary

  • ASN.1 defines the structure (which fields exist and how they relate).
  • DER takes that structure and turns it into a sequence of bytes following strict rules.
  • The names of fields are in the ASN.1 schema, not in the encoded data.
  • A decoder needs the ASN.1 schema to interpret the DER-encoded bytes correctly.