An Evaluation Study on Compression Techniques in Hadoop

File Size:
1.05 MB
Volume 3, Issue 2 (February, 2017)
Publication No:
Ramanjot kaur, Dr. Raman Chadha, Nitin Verma
12 x

Now a day, big data is widely used term in Information Technology. It contains vast amount of data. This data may be in any type e.g. structured, unstructured or semi structured. Sources of big data have different kind of features like frequency, volume, velocity and veracity of the data. Growth in the volume of data may be because of growth in use of phones, internet, social media or other technical devices. And also it is very difficult to handle big data in efficient and correct way and it may have various challenges and difficulties. Traditional data handling techniques like DBMS and RDBMS are not able to handle big data as it has a very large size and different forms. As the amount is very large, and main challenge is that how we store this massive amount of data. To solve storage problem we can use compression of data. Compression will surely add the advantage to store this large size. But there are also various issues when using compression techniques. A biggest issue is whether the compression is not as in hadoop the data is stored into spllitable form so it is important that the compression technique also a in a spllitable form. Different types of compression algorithm we will implement in this paper and we show which is more efficient than the others. So the problem in using the compression in hadoop is that out of all the different compression techniques which one can be use for a reasonable and fast compression. So in this paper we are going to represent various compression techniques and finally a comparison between them. [1] It may help to choose an efficient compression technique for data compression in hadoop where data is stored into a split manner. So the main aim of this paper is to show the practical results of their compression so that one can choose the best for their data.

Hadoop, Compression, Big Data


Tags Associated: Big Data Hadoop Compression