How to crack common Big Data interview questions?

Big Data has become a wonderful opportunity for people keen on getting into IT jobs in the USA. Today, extraordinary amounts of data is being harvested and processed, thanks to faster computing. Big data analytics is about employing specialized software and tools to carryout analysis and visualization of the vast amounts of harvested data. There is no dearth of opportunities for those skilled in Big Data Analytics. Yet, the job market for BigData Jobs in USA is extremely competitive. Staying abreast on the common Big Data interview questions and answers goes a long way towards beating this competition. Here’s a look at some of the questions to expect at interviews.

Q: What is Hadoop?

A: Hadoop is an open source programming framework based on Java. A branch of Apache Software Foundation’s Apache project, it is employed in the distributed processing and storage of big data sets.

Q: What are the key Hadoop tools that augment Big Data performance?

A: Some of the many tools that work well for perking up Big Data performance include Apache Hadoop, Apache Ambari, HDFS (Hadoop Distributed File System), Apache HBase, Apache Hive, Apache Sqoop, Apache Pig, ZooKeeper, NoSQL, Apache Mahout, Apache Lucene/Apache Solr, Apache Avro, among others. (Of course, you will have to brush up on the purpose of these tools).

Q: What is the small file problem in HDFS?

A: A small file is something that’s a lot smaller compared to the HDFS block size, which is 64MB by default. Most Hadoop users have loads of files to process – even the smaller ones. This issue is that the HDFS doesn’t have the capacity to manage lots of files. In HDFS, each file, directory and block is symbolized as an object in the namenode’s memory. These normally require 150 bytes each. When there are ten million files, with each occupying a block, 3 GB is used. The HDFS cannot handle more than this at the moment and is not good for accessing small files. In simple words, the HDFS is meant only for enabling streaming access of large files.

Q: What is the best hardware configuration to run Hadoop?

Configuration usually depends on workflow requirements. Dual core machines or dual processors with 4 to 8GB RAM are ideal. ECC memory is required to avoid checksum errors.

Q: Can you list the common Input Formats in Hadoop?

A: The Input formats are Key value Input format, Text Input format and Sequence file Input format.

Q: What is TaskInstance?

A: It is a specific Hadoop MapReduce work process that runs on any particular slave node. In order to perk up performance, each task instance comes with its very own JVM process.

Q: What is the use of counters in Hadoop?

A: Counters are employed in Hadoop for collecting statistics from a job carried out using MapReduce. They keep track of events and collect job statistics like the count of rows read, number of rows written as output and so on.

Q: How is Hadoop code debugged?

A: DeBuggiug is done with a web interface offered by Hadoop framework and through the use of Counters.

Q: How do you check file systems?

A: The "fsck" command is used to carry out checks to block names and locations, and also evaluate the health condition of the file system.