Monday, March 23, 2015

Cloud - The Internet Revolution


All around us we hear people speaking about cloud this and cloud that. Apple came up with iCloud. So what is this "cloud".

A nice article I read a few days back.
http://gizmodo.com/what-is-the-cloud-and-where-is-it-1682276210

Hopefully, this will make things a little clear.

Friday, March 20, 2015

Setting up multiple MySQL Instances in the same machine

  1. Download the MySQL server tarball
cd /mnt/softwares/

  1. Unzip and install into local system and create Master-Slave Configuration
gunzip mysql-5.6.23-linux-glibc2.5-x86_64.tar.gz
tar -xf mysql-5.6.23-linux-glibc2.5-x86_64.tar
groupadd mysql
useradd -r -g mysql mysql
ln -s mysql-5.6.23-linux-glibc2.5-x86_64 mysql
cd /mnt/softwares/mysql
chown -R mysql .
chgrp -R mysql .
scripts/mysql_install_db –user=mysql
chown -R root .
chown -R mysql data

vi /etc/my.cnf
 
#ADD BELOW LINES TO my.cnf >>>
 [mysqld]
 server-id=1
 bind-address=127.0.0.1
 character-set-server= utf8
 log-bin=mysql-bin
 log-slave-updates=1
 binlog-format=row
 gtid-mode=on
 enforce-gtid-consistency=1
 innodb-file-per-table=1
 replicate-ignore-db=mysql
 datadir=/var/lib/mysql3306
 socket=/var/lib/mysql/mysql3306.sock
 port=3306
 explicit_defaults_for_timestamp=1

 sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES

 [mysqld_safe]
 log-error=/var/log/mysqld3306.log

 [client]
 default-character-set= utf8


scripts/mysql_install_db –user=mysql –dataDir=/var/lib/mysql3306
vi /etc/my2.cnf
 
#ADD BELOW LINES TO my2.cnf >>>
 [mysqld]
 server-id=2
 bind-address=127.0.0.1
 character-set-server= utf8
 log-bin=mysql-bin
 log-slave-updates=1
 binlog-format=row
 gtid-mode=on
 enforce-gtid-consistency=1
 innodb-file-per-table=1
 replicate-ignore-db=mysql
 datadir=/var/lib/mysql3307
 socket=/var/lib/mysql/mysql3307.sock
 port=3306
 explicit_defaults_for_timestamp=1

 sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES

 [mysqld_safe]
 log-error=/var/log/mysqld3307.log

 [client]
 default-character-set= utf8
 

scripts/mysql_install_db –user=mysql –dataDir=/var/lib/mysql3307

  1. Start both MySQL instances
bin/mysqld_safe --defaults-file=/etc/my.cnf &
bin/mysqld_safe --defaults-file=/etc/my2.cnf &

And you are done.

Wednesday, March 18, 2015

MySQL - mysqlbinlog default-character-set error



I tried reading the bin log file in mysql. What I was greeted with is this error.

shell> bin/mysqlbinlog /var/lib/mysql3306/mysql-bin.000001
bin/mysqlbinlog: unknown variable 'default-character-set=utf8'

Solving this is pretty simple.

shell> bin/mysqlbinlog --no-defaults -v /var/lib/mysql3306/mysql-bin.000001

And done.

Monday, March 16, 2015

Connecting to MySQL as a slave ( Open Replicator )

Open Replicator is a open source library provided here.

It is a very useful and easy to use tool to get the bin log events fired when the MySQL masters writes into the bin log file.

Usage as per the Open Replicator wiki:
final OpenReplicator or = new OpenReplicator();
or.setUser("root");
or.setPassword("123456");
or.setHost("localhost");
or.setPort(3306);
or.setServerId(6789);
or.setBinlogPosition(4);
or.setBinlogFileName("mysql_bin.000001");
or.setBinlogEventListener(new BinlogEventListener() {
    public void onEvents(BinlogEventV4 event) {
        // your code goes here
    }
});
or.start();

One thing to note is that the open replicator works in two ways that can be set via the my.cnf file in MySQL. They are;

1. Statement based replication: In this case only QueryEvents are fired and the data is sent in the form of sql query statements. For this the binlog-format needs to be set to "STATEMENT"

2. Row based replication: In this case different events are fired for different statements used i.e. an UpdateEvent is fired for update statements, WriteEvent is fired for Insert statements and DeleteEvent is fired for Delete statements.

Services like facebook, linkedin-databus, flipkart, to name a few, have used this powerful library to manage mysql data.

Saturday, March 7, 2015

Design Patterns - An eye for similarities - Part 4

Factory Pattern - Lets keep churning

Of all the patterns, I think the factory pattern is one of the patterns that is most used. Nearly everyone must have gone by a usage of this with or without realizing it.

In a real world scenario, a factory can produce a number of Products. Each of this product will have their own way of making it, but they all fall under the general idea of the factory. So, whenever the user wants some product, the factory manager just makes that product and passes it to the user.

The internal implementation is kept hidden. So the user does not have to know how the way in which the instance is given, whether a pool is kept or it is created as and when a request comes and so on and so forth. The only thing that the user knows is that they will get a fresh instance every time they ask for it.

The diagram below explains this process.


Usage:

ShapeFactory sf = new ShapeFactory();
Shape circle = shapeFactory.getShape("circle");
circle.draw();

So, the users essentially do not need to know what is happening inside the factory as long as they get the shape that they asks for.


Related:
Part 2 [ Strategy and Observer Pattern ]
Part 3 [ Decorator Pattern ]





Tuesday, March 3, 2015

Hide Desktop in CentOS


I am normally an advocate of a clear desktop. So, I do not want to push anything on the desktop. However, inevitably something or the other creeps up and we push things on to the desktop. Which is why I keep the desktop invisible.

In Windows system this is pretty simple to achieve with a right-click and "Show/Hide Desktop"

Fortunately, it is pretty simple in Linux OS too. Just uncheck the "Show Desktop" in nautilus/app and lo and behold. :)

Saturday, February 28, 2015

Getting data from HBase in Java



1. Make sure that the HBase configuration file is in your classpath when you run the program

                /**
* Create a HBase Table for employees
*/
// Create a HBase Configuration
Configuration conf = HBaseConfiguration.create();
// Create an instance of the HBaseAdmin to connect to HBase
HBaseAdmin admin = new HBaseAdmin(conf);
// Create a table
HTableDescriptor tableDescriptor = new HTableDescriptor(TableName.valueOf("employee"));
tableDescriptor.addFamily(new HColumnDescriptor("personalinfo"));
tableDescriptor.addFamily(new HColumnDescriptor("workinfo"));
admin.createTable(tableDescriptor);
/**
* Add a record to the table
*/
// Get the Table instance
HTable table = new HTable(conf, "employee");
// Create a Row with key as combination of name, department, subdepartment, and employee number
Put put = new Put(Bytes.toBytes("Sam-IT-Infrastructure-43"));
// Add data to the families in the row
put.add(Bytes.toBytes("personalinfo"), Bytes.toBytes("firstName"), Bytes.toBytes("Sam"));
put.add(Bytes.toBytes("personalinfo"), Bytes.toBytes("lastName"), Bytes.toBytes("Witwicky"));
put.add(Bytes.toBytes("personalinfo"), Bytes.toBytes("gender"), Bytes.toBytes("M"));
put.add(Bytes.toBytes("workinfo"), Bytes.toBytes("email"), Bytes.toBytes("samwitwicky@abcorp.com"));
put.add(Bytes.toBytes("workinfo"), Bytes.toBytes("experience"), Bytes.toBytes("5"));
table.put(put);
table.flushCommits();
table.close();
/**
* Update a record in the table
*/
// Get the Table instance
table = new HTable(conf, "employee");
// The row with key as combination of name, department, subdepartment, and employee number
Put updatePut = new Put(Bytes.toBytes("Sam-IT-Infrastructure-43"));
// Update experience and add designation
updatePut.add(Bytes.toBytes("workinfo"), Bytes.toBytes("experience"), Bytes.toBytes("6"));
updatePut.add(Bytes.toBytes("workinfo"), Bytes.toBytes("designation"), Bytes.toBytes("Senior Designer"));
table.put(updatePut);
table.flushCommits();
table.close();
/**
* Delete a record in the table
*/
// Get the Table instance
table = new HTable(conf, "employee");
// Delete the row
Delete delete = new Delete(Bytes.toBytes("Sam-IT-Infrastructure-43"));
table.delete(delete);
table.flushCommits();
table.close();