Friday, July 4, 2014

Installing Google Chrome in CentOS 6.5


The problem:

Seems a hassle to install chrome browser on CentOS because Google it seems has dropped support for CentOS 6.

The solution:

I found this lovely piece of advice by Richard Lloyd and worked like a charm for me.

# wget http://chrome.richardlloyd.org.uk/install_chrome.sh
# chmod u+x install_chrome.sh
# ./install_chrome.sh 
 
Cheers.

Tuesday, July 1, 2014

ChecksumException in HDFS


Came across this exception a couple of times.

/mnt/softwares/hadoop/bin/hadoop fs -copyFromLocal /mnt/folder1/folder2/file.tsv /mnt/folder1/folder2/file.tsv
14/07/01 09:47:58 INFO fs.FSInputChecker: Found checksum error: b[0, 0]=
org.apache.hadoop.fs.ChecksumException: Checksum error: /mnt/folder1/folder2/file.tsv at 0
        at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readChunk(ChecksumFileSystem.java:219)
        at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
        at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
        at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
        at java.io.DataInputStream.read(DataInputStream.java:83)
        at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:68)
        at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:47)
        at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:100)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:230)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:176)
        at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1209)
        at org.apache.hadoop.fs.FsShell.copyFromLocal(FsShell.java:138)
        at org.apache.hadoop.fs.FsShell.run(FsShell.java:1950)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.fs.FsShell.main(FsShell.java:2090)
copyFromLocal: Checksum error: /mnt/folder1/folder2/file.tsv at 0

Seems like it is a bug reported into hadoop

Simple solution: Remove the crc file from the folder from where you are copying the file.

Cheers.

Wednesday, June 25, 2014

Filter query in HBase Shell



For a quick reference, to filter a column in HBase Shell, we can run the command as below,

import org.apache.hadoop.hbase.filter.CompareFilter
import org.apache.hadoop.hbase.filter.SingleColumnValueFilter
import org.apache.hadoop.hbase.filter.SubstringComparator
import org.apache.hadoop.hbase.util.Bytes
scan 'table_name', {FILTER => SingleColumnValueFilter.new(Bytes.toBytes('family'),Bytes.toBytes('qualifier'),CompareFilter::CompareOp.valueOf('EQUAL'),SubstringComparator.new('filter_value')), COLUMNS => ['family:qualifier']}

Hope this helps.
Cheers.

Saturday, June 21, 2014

Case Insensitive Check in List



Java does not provide a ignore case comparison of Strings in a List. I have a pretty simple implementation for the same,

public static boolean listContainsIgnoreCase(List listToCheck, String stringToCheck)
{
for (String s : listToCheck) 
{
       if (stringToCheck.equalsIgnoreCase(s)) return true;
   }
return false;
}

Cheers.