Archive for category Uncategorized

Encryption reference project

This project provides reference implementation for encryption decryption in Java. Only basic Java 8 features are used with no extra libraries.

The goal is to have a reference implementation how to encrypt / decrypt data properly using basic JRE tools, at this time neglecting encryption speed, hashing speed, etc.. Effectively this approach should be valid in all programming languages or tools.

The code snippets shown reference best practices without explaining why (such as explaining ciphertext maleability, etc..)

Lowest feasible settings are used (AES-128, SHA-256, 1024-bit RSA, RSA-PKCS1) feasible for users with Java 7 and without Unlimited Strength JCE Policy.

For simplification:

  • only constant string is used (no streaming, updates, ..)
  • only block ciphers are used
  • inputs and outputs are stored only in a Java class, so we don’t deal with file handling, etc..

Reference use cases in this project:

  • simple symmetric encryption and decryption
  • password symmetric encryption and decryption
  • asymmetric encryption

There are more ways and use cases, but this is typically used most often I see. What is not implemented (maybe it will come later)

  • stream ciphers
  • CTR mode (allowing partial decryption)

Rules to follow

  • don’t invent your own crypto !!!!!!!
  • password is not key
  • cut one of your fingers for each time you reuse a nonce
  • treat unauthenticated ciphertext as nuclear waste

Encryption use cases

Parameter used in this example:

 // parameters for AES-128 with block size 128 bit
 private static final String SYMMETRIC_CIPHER_NAME = "AES/CBC/PKCS5Padding";
 private static final String SYMMETRIC_KEY_ALG = "AES";
 private static final int SYMMETRIC_BLOCK_SIZE = 128;
 // key size for different ciphers
 private static final byte[] SYMMETRIC_KEY = Base64.getDecoder().decode("4Y6kfyPmosh1I6dSkMEk1Q==");
 private static final String TEXT_PASSWORD = "some long and complex password";
 private static final String PBKDF_ALG = "PBKDF2WithHmacSHA256";
 private static final int PBKDF_INTERATIONS = 800000;
 private static final String HASH_ALGORITHM_NAME = "HmacSHA256";
 private static final String PBE_CIPHER_NAME = "PBEWithHmacSHA256AndAES_128";
 
 private static final String PKI_CIPHER_ALG = "RSA/ECB/PKCS1Padding";

 

Simple symmetric encryption

Encryption:

 // initialization vector
 SecureRandom rnd = new SecureRandom();
 byte[] iv = new byte[SYMMETRIC_BLOCK_SIZE / 8];
 rnd.nextBytes(iv);
 encryptionParams.setIv(iv);

 IvParameterSpec ivParamSpec = new IvParameterSpec(iv);
 SecretKey symmetricKey = new SecretKeySpec(encryptionParams.getKey(), SYMMETRIC_KEY_ALG);

Cipher cipher = Cipher.getInstance(SYMMETRIC_CIPHER_NAME);
cipher.init(Cipher.ENCRYPT_MODE, symmetricKey, ivParamSpec);

 // for HMAC we should be able to use the same key as for encryption
 // for CBC-MAC it may not be the case
 // https://en.wikipedia.org/wiki/CBC-MAC#Using_the_same_key_for_encryption_and_authentication
 Mac mac = Mac.getInstance(EncryptionTest.HASH_ALGORITHM_NAME);
 mac.init(symmetricKey);

 byte[] encrypted = cipher.doFinal(encryptionParams.getPlaintext());
 encryptionParams.setCiphertext(encrypted);
 byte[] authTag = mac.doFinal(encrypted);
 encryptionParams.setMac(authTag);

LOGGER.log(Level.INFO, "Input encrypted, tag value: {0}, ciphertext: {1}",
 new Object[]{
 Base64.getEncoder().encodeToString(authTag),
 Base64.getEncoder().encodeToString(encrypted)
 })

 

Decryption:

IvParameterSpec ivParamSpec = new IvParameterSpec(encryptionParams.getIv());
 SecretKey symmetricKey = new SecretKeySpec(encryptionParams.getKey(), SYMMETRIC_KEY_ALG);

Cipher cipher = Cipher.getInstance(SYMMETRIC_CIPHER_NAME);
cipher.init(Cipher.DECRYPT_MODE, symmetricKey, ivParamSpec);

Mac mac = Mac.getInstance(EncryptionTest.HASH_ALGORITHM_NAME);
mac.init(symmetricKey);

 byte[] computedMac = mac.doFinal(encryptionParams.getCiphertext());
 LOGGER.log(Level.INFO, "computed mac: {0}", Base64.getEncoder().encodeToString(computedMac));

 if(this.compareTags(computedMac, encryptionParams.getMac())) {
 byte[] decrypted = cipher.doFinal(encryptionParams.getCiphertext());
 LOGGER.log(Level.INFO, "decrypted text: {0}", new String(decrypted,"UTF-8"));

 } else {
 LOGGER.log(Level.WARNING, "Decryption failed, authentication tag doesn't match");
 throw new IllegalArgumentException("authentication failed");
 }

 

There is a list of ciphers required to be supported by any Java implementation, see the Cipher javadoc

One of the ciphers is chosen: AES-128 CBC mode (AES/CBC/PKCS5Padding) as is guaranteed to be available and includes the padding (we don’t have to deal with it separately)

The test application will print a whole list of available services (ciphers, modes, ..) however they are not guaranteed to be available on different versions, systems or environments.

A big list of other features (cipher, keys, hashes, ..) is available with the Bouncy Castle project.

Lessons to be learned:

  • use random IV (initialization vector / nonce)
  • use authenticated encryption
  • key must have fixed size depending on cipher used
  • data are encrypted in blocks (with block ciphers), for simplification a cipher with implicit PKCS #5 padding is used, otherwise we will have to implement the padding on our own.

Key size:

  • AES – 128, 192 or 256 bits
  • 3DES (DESede) – 168 or 112 bits (according to OpenSSL, effective key strength is 80 bits, so 3DES is considered as a weak cipher)
  • DES – 56 bit (too short for current times, use only for backward compatibility)
  • Blowfish – 32–448 bits
  • ARCFOUR – 40–2048 bits
  • RC2 – 8–1024 bits

Password symmetric encryption

Password based encryption (PBE) is the same as previous symmetric encryption, just key key is based (derived) from passwords.

The issues with user-entered (and remembered) passwords are

  • passwords tend to have low entropy (are not long and random enough) and often are feasible for dictionary or brute-force attack
  • password length does not match required key length

Therefore the encryption keys are derived from the password using the PBKDF (Password-Based Key Derivation Function).

PBKDF will output a key with required length using salt and iteration count parameters. The password salt (nonce) will make the output “randomized” (so even when reusing the password the key will be different). The iteration count should make brute-forcing and dictionary attack less feasible.

 

Encryption:

 // create salt
 SecureRandom rnd = new SecureRandom();
 // salt can have arbitrary length
 byte[] psswdSalt = new byte[SYMMETRIC_BLOCK_SIZE]; 
 rnd.nextBytes(psswdSalt);
 params.setSalt(psswdSalt);
 
 // create key from password
 SecretKeyFactory secKeyFactory = SecretKeyFactory.getInstance(PBKDF_ALG);
 KeySpec pbeSpec = new PBEKeySpec(password.toCharArray(), psswdSalt, PBKDF_INTERATIONS, SYMMETRIC_KEY.length*8);
 SecretKey pbeSecretKey = secKeyFactory.generateSecret(pbeSpec);
 SecretKey secKey = new SecretKeySpec(pbeSecretKey.getEncoded(), SYMMETRIC_KEY_ALG);
 params.setKey(secKey.getEncoded());
 this.symmetricEncryption(params);

 

or using the PBE (password based encryption) cipher (PBEWithHmacSHA256AndAES_128)

// create salt, salt can have arbitrary length
 byte[] psswdSalt = new byte[SYMMETRIC_BLOCK_SIZE]; 
 rnd.nextBytes(psswdSalt);
 params.setSalt(psswdSalt);
 
 byte[] iv = new byte[SYMMETRIC_BLOCK_SIZE / 8];
 rnd.nextBytes(iv);
 params.setIv(iv); 
 IvParameterSpec ivParamSpec = new IvParameterSpec(iv);
 
 PBEParameterSpec pbeParamSpec = new PBEParameterSpec(psswdSalt, PBKDF_INTERATIONS, ivParamSpec);
 PBEKeySpec pbeKeySpec = new PBEKeySpec(password.toCharArray());
 SecretKeyFactory pbeKeyFactory = SecretKeyFactory.getInstance("PBEWithHmacSHA256AndAES_128");
 SecretKey pbeKey = pbeKeyFactory.generateSecret(pbeKeySpec);
 
 Cipher cipher = Cipher.getInstance(PBE_CIPHER_NAME);
 cipher.init(Cipher.ENCRYPT_MODE, pbeKey, pbeParamSpec);

 Mac mac = Mac.getInstance(EncryptionTest.HASH_ALGORITHM_NAME);
 mac.init(pbeKey);

byte[] encrypted = cipher.doFinal(params.getPlaintext());
 params.setCiphertext(encrypted);
 byte[] authTag = mac.doFinal(encrypted);
 params.setMac(authTag);

Decryption:

byte[] psswdSalt = encryptionParams.getSalt(); 
 byte[] iv = encryptionParams.getIv();
 
 IvParameterSpec ivParamSpec = new IvParameterSpec(iv);
 PBEParameterSpec pbeParamSpec = new PBEParameterSpec(psswdSalt, PBKDF_INTERATIONS, ivParamSpec);
 PBEKeySpec pbeKeySpec = new PBEKeySpec(password.toCharArray());
 SecretKeyFactory pbeKeyFactory = SecretKeyFactory.getInstance("PBEWithHmacSHA256AndAES_128");
 SecretKey pbeKey = pbeKeyFactory.generateSecret(pbeKeySpec);
 
 
 Cipher cipher = Cipher.getInstance(PBE_CIPHER_NAME);
 cipher.init(Cipher.DECRYPT_MODE, pbeKey, pbeParamSpec);

Mac mac = Mac.getInstance(EncryptionTest.HASH_ALGORITHM_NAME);
 mac.init(pbeKey);
 
 byte[] computedMac = mac.doFinal(encryptionParams.getCiphertext());
 LOGGER.log(Level.INFO, "computed mac: {0}", Base64.getEncoder().encodeToString(computedMac));
 
 if(this.compareTags(computedMac, encryptionParams.getMac())) {
 byte[] decrypted = cipher.doFinal(encryptionParams.getCiphertext());
 LOGGER.log(Level.INFO, "decrypted text: {0}", new String(decrypted,"UTF-8"));
 
 } else {
 LOGGER.log(Level.WARNING, "Decryption failed, authentication tag doesn't match");
 throw new IllegalArgumentException("authentication failed");
 }

 

Asymmetric encryption

Asymmetric encryption is based on fact that for encryption and decryption there are different keys used. In this example we will use RSA keypair stored in the JKS keystore

Asymmetric encryption is feasible for encryption of data much smaller than the asymmetric key length, therefore for encrypting larger chunk of data only a random symmetric key is encrypted. The symmetric key is used to encrypt the data.

 

Read the keystore entry

 private KeyStore.Entry getKeystoreEntry() throws 
 KeyStoreException, NoSuchAlgorithmException, 
 UnrecoverableEntryException, IOException, CertificateException {
   KeyStore keystore = KeyStore.getInstance("JKS");
   try (InputStream in = new FileInputStream(KEYSTORE_FILE)) {
     keystore.load(in, KEYSTORE_FILE_PASSWORD.toCharArray());
   return keystore.getEntry(KEYSTORE_ALIAS, new KeyStore.PasswordProtection(KEYSTORE_KEY_PASSWORD.toCharArray()));
   }
 }

 

Encryption using  “RSA/ECB/PKCS1Padding”

 // generate random key without revealing the real key bytes
 // some HSM (hardware security modules) won't allow to set or get bytes
 // revealing the real key value
 KeyGenerator keyGenerator = KeyGenerator.getInstance(SYMMETRIC_KEY_ALG);
 SecretKey symmetricKey = keyGenerator.generateKey();
 
 // this assumes there's whole keypair (including private key)
 // normally only a certificate with PubKey is available
 KeyStore.PrivateKeyEntry privKeyEntry = (KeyStore.PrivateKeyEntry) this.getKeystoreEntry();
 PublicKey pubKey = privKeyEntry.getCertificate().getPublicKey();

 params.setKey(symmetricKey.getEncoded());
 // execute symmetric encryption
 this.symmetricEncryption(params);
 // encrypt the key with the public key
 Cipher cipher = Cipher.getInstance(PKI_CIPHER_ALG);
 cipher.init(Cipher.WRAP_MODE, pubKey);
 byte[] wrappedKey = cipher.wrap(symmetricKey);
 LOGGER.log(Level.INFO, "Wrapped key: {0}", Base64.getEncoder().encodeToString(wrappedKey));
 params.setKey(wrappedKey);

 

Decryption

 byte[] encryptedKey = params.getKey();
 
 KeyStore.PrivateKeyEntry privKeyEntry = (KeyStore.PrivateKeyEntry) this.getKeystoreEntry();
 PrivateKey privKey = privKeyEntry.getPrivateKey();
 
 // decrypt the key with the private key
 Cipher cipher = Cipher.getInstance(PKI_CIPHER_ALG);
 cipher.init(Cipher.UNWRAP_MODE, privKey);
 Key symmetricKey = cipher.unwrap(encryptedKey, SYMMETRIC_KEY_ALG, Cipher.SECRET_KEY);
 params.setKey(symmetricKey.getEncoded());
 
 // execute symmetric decryption
 this.symmetricDecryption(params);

Resources

Other useful resources over cryptography

Advertisements

Leave a comment

PostgreSQL continuous backup

This post intents to serve as my personal notebook, when someone know an improvement, feel free to comment.

A client is having a single PostgreSQL server instance to store production data (hosted by a big IaaS provider) with VPN access to the hosted network. Now the question is – how  could the client backup the database data and restore the database in case of disaster? Many serious DB admins may hate this approach but the client has decided going this way instead of a proper DB cluster (or having the DB ran as PaaS as we adviced)

My idea is – to let the client backup (rsync) the database WAL files.

Setting up the backup

/var/lib/pgsql/9.6/data/pg_hba.conf

local replication postgres peer

/var/lib/pgsql/9.6/data/postgresql.conf

wal_level = archive
max_wal_senders = 1
archive_mode = on
archive_command = 'test ! -f /var/lib/pgsql/9.6/backups/%f && gzip < %p > /var/lib/pgsql/9.6/backups/%f'

Restart PostgreSQL
Create a base backup:

pg_basebackup -D /some/backup/folder -Ft -x -z

Here we have to assume the client can fetch (rsync) the base backup and incremental WAL log files from the backup location /var/lib/pgsql/9.6/backups/

 

Using selinux (e.g. RHEL), the postgres service require special permissions for directories to write to. Therefore if we want to backup to different folder (/mnt/shared/dbbackups), we have to add following permissions to the folder:

chcon system_u:object_r:postgresql_db_t:s0 /mnt/shared/dbbackups

 

 

Restore from backup

Restore the base backup (assuming the WAL log files are in the location /media/backup/wals

Reinstall postgresql (in the stopped state)

Restore the base backup

Create a recovery file: /var/lib/pgsql/9.6/data/recovery.conf

restore_command = 'gunzip < /media/backup/wals/%f > %p'

Restart PostgreSQL server

Leave a comment