Installing Hadoop-LZO compression module RPMs
Recently I wrote about compiling and installing LZO support for Hadoop.
But now I found RPMs for this by Cloudera. Strangely they arent mentioned anywhere but here “Installing-and-Using-Impala” . So its much simpler to install now.
# this is for RHEL/CentOS . For Debian, Ubuntu, SLES/SUSE see http://archive.cloudera.com/gplextras/
cd /etc/yum.repos.d/ && wget http://archive.cloudera.com/gplextras/redhat/6/x86_64/gplextras/cloudera-gplextras4.repo
yum install hadoop-lzo-cdh4 hadoop-lzo-cdh4-mr1
cd /etc/yum.repos.d/ && wget http://archive.cloudera.com/gplextras/redhat/6/x86_64/gplextras/cloudera-gplextras4.repo
yum install hadoop-lzo-cdh4 hadoop-lzo-cdh4-mr1
Its really as simple as that! Installs LZO for MapReduce and for Hadoop to /usr/lib/hadoop/lib/ and /usr/lib/hadoop-0.20-mapreduce/lib and also to the native/ paths.
Check my older blog post about the necessary configuration settings which are left to do. Just skip the compilation part.
No comments yet.