Hadoop Credential API

In Hadoop 2.6 a fundamental feature was introduced that did not get much attention but will play an important role moving forward – the Credential API. Looking ahead the Credential Management Framework (CMF) will play an important role for the pluggable token authentication framework, column encryption in ORC files, or the transparent data encryption. But not only future components but applications build for Hadoop can benefit from it.

Background

Many applications around Hadoop store passwords in configuration files in clear text format. This is often not acceptable for most users that share security concerns. Many application like sqoop have a very limited approach to storing sensitive database credentials.

For long developers had to come up with their own solution for their implementations. Already existing mechanism were not supported and still aren’t even with the Credential API.

  1. Password Indirection or Aliasing
  2. Management of identity and trust keystores
  3. Rolling of key pairs and credentials
  4. Discovery of externally provisioned credentials
  5. Service specific CMF secret protection
  6. Syntax for Aliases within configuration files

How it works

Store credentials in in keystore:

user:///,jceks://file/tmp/test.jceks,jceks://hdfs@nn1.example.com/my/path/test.jceks

Use CLI to get and store credentials:

Example: -provider jceks://file/tmp/test.jceks

Usage: hadoop credential <subcommand> [options]

Use the Java API:

import org.apache.hadoop.security.alias.CredentialProvider;
import org.apache.hadoop.security.alias.CredentialProviderFactory;
import org.apache.hadoop.conf.Configuration;

CredentialProvider.CredentialEntry credEntry = provider.getCredentialEntry(alias);

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s