Update Java to JDK 8 on Amazon Elastic MapReduce

Update Java to JDK 8 on Amazon Elastic MapReduce

Last updated:

The new emr-4.4.0 label on Amazon Elastic MapReduce now supports Java 8.

If you are running older labels, you'll have to download it yourself (sudo yum install java-1.8.0 java-1.8.0-openjdk-devel)

First off, make sure your cluster has been created with a private key, otherwise you can't SSH into it:

More info on how to SSH into your cluster

Once you are in, type java -version to see what you current version is:

[hadoop@ip-000-00-00 ~]$ java -version
java version "1.7.0_95"
OpenJDK Runtime Environment (amzn-2.6.4.0.65.amzn1-x86_64 u95-b00)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)

If you see a version starting with 1.7 (as in the example above) it means you're running Java 7.

The solution

To change it to Java 8:

$ sudo alternatives --config java

You will be given an option of what JDK you want. Type 2 to select Java 8 and then hit ENTER.

Checking that it worked

[hadoop@ip-000-00-00 ~]$ java -version
openjdk version "1.8.0_71"
OpenJDK Runtime Environment (build 1.8.0_71-b15)
OpenJDK 64-Bit Server VM (build 25.71-b15, mixed mode)

Trobleshooting

If your Spark/Hadoop Jobs don't seem to be picking up the new Java version, you may also need to alter $JAVA_HOME in some configuration files, e.g. Spark's and Hadoop's:

$ echo "export JAVA_HOME=/usr/lib/jvm/java-1.8.0" | sudo tee -a /usr/lib/spark/conf/spark-env.sh
$ echo "export JAVA_HOME=/usr/lib/jvm/java-1.8.0" | sudo tee -a /usr/lib/hadoop/etc/hadoop/hadoop-env.sh

See also