Things to remember : In Map Reduce

Q 1. What is IdentityMapper?
A - An empty Mapper which directly writes key/value to the output.
         Mapper<K,V,K,V>
Q 2. What is InverseMapper?
A - A Mapper which swaps the <Key,Value> to <Value,Key>.
         Mapper<K,V,V,K>
Q 3. What is IdentityReducer?
A - It performs no reduction, directly writes key/value to the output.
         Reducer<K,V,K,V>
Q 4. What is Partitioner?
A - It runs after completion of Map Jobs. A custom Partitioner can be implemented to decide which key/value should go to which Reducer.
In Map-Reduce model, unique key 'K' with all its Iterable<V> should go to same Reducer.
Q 5. What are the uses of Combiner?
A - It helps in performing local aggregation on Map jobs output to reduce the ammount of data sent to any Reducer.
Q 6. Where Map outputs are stored?
A - Intermediate or Grouped Map output are stored in Sequence File(can be gzipped) on HDFS cluster.
Q 7. How to set number of mapper & reducer?
A - JobConf class object is used to set number of mapper and reducer.
JobConf is present in package org.apache.hadoop.mapred and extends org.apache.hadoop.conf.Configuration
public void setNumMapTasks(int n);// sets number of mapper Job
public void setNumReduceTasks(int n);// sets number of reducer Job
Q 8. What is ChainMapper?
A - It allows to use multiple Mapper class in single Map task.
Output of one mapper is passed to another mapper and so on.
Each Mappper get executed in chain.
Q 9. What is RegexMapper?
A - A Mapper that extracts text matching a regular expression.

Things to remember : In Core JAVA

Q 1. Can you tell, which Algorithm is used by HashMap/HashTable?
A - HashMap internally uses bucket to store key-value pair. When a key is passed to HashMap, it is not used as 'key' as it is! It gets converted to another HashKey using HashCode(). When same HashKey is generated for multiple key(s) (ie: Collision in HashMap/HashTable), It(another key-value pair) get stored in same bucket as next item( Each bucket is a Linked List, It can contain multiple key-value pair).
HashMap can take a 'initial Capacity' & 'load Factor' in its constructor. 
initial Capacity : number of bucket get created at the time of initialization. 
load Factor : number of buckets get increased when Items cross this load factor.
HashTable is a synchronized version of HashMap. But HashMap gives performance bonus as object is not accessed by multiple Threads. 
Q 2. Name some way of Inter Process Communication(IPC)?
A - These are :

1. Socket
2. Message Queue
3. Pipe
4. Signal
5. File
6. Remote Method Invocation (RMI)
7. Shared Memory
8. SOAP, REST, Thrift, XML, JSON

Q 3. What is Mutual Exclusion?

A - Mutual Exclusion in OS (Mutex) is a collection of techniques/algorithms for sharing resources so that concurrent uses do not conflict and cause unwanted interactions. One of the most commonly used techniques for mutual exclusion is the semaphore.
Q 4. What are Abstraction and Encapsulation?
A - Abstraction : Hiding away unimportant details of an object, focuses on outside view.
      Encapsulation : It is defined as the process of wrapping up the data members and member functions together into a single unit.

What is : Hadoop Sequence File?

Hadoop Sequence File : These are flat files consisting of binary Key-Value Pair. It can store any key-value pair as byte arrays.

3 Types : 

UnCompressed

Record-Compressed

Block-Compressed.

Need for Sequential File : Hadoop is meant for processing BigData. It has 64 MB default block size on any cluster. Which mean any file with size lesser than 64 MB will eventually occupy 64 MB physical space on disk storage.

      In practical, Applications deal with files with fewer KB. So, It is advantageous to keep number of such small file in sequential Key-Value pair, which allows programmers to run similar logic on each file found in a block with help of Map-Reduce Jobs.

    -org.apache.hadoop.io.SequenceFile Class provides Read, Write methods. It also grants provision for 'Sorting' of SequenceFile Keys.

Thank YOU

How to : choose between DOM, SAX or XMLStreamWriter

What is XML : I call it a "language for Internet", It help applications to communicate  seamlessly. At the same time it's in human readable format too. Any XML file typically contains 'Elements' and 'Attributes', which are also called XML 'Node'. 
(Element, CData, Comment, Attribute, Entity, Text are few examples of Node Type).
There are number of API(s) available to work with XML, and each has its positive and negative aspects.

W3C DOM : 

good for - random read and  write with XML nodes.
not suitable for - larger memory footprint, performance.

SAX :

good for : faster read access, it's lightweight.
not suitable for - writing/creating XML nodes.

XMLStreamWriter :

good for - streaming out XML while building it, useful in web services handling larger files.
not suitable for - random read or write, It's sequential, one-way, cursor like implementation. 


All what you need is to choose your API closest to your need. XMLStreamWriter is a good for all purpose. Most effective for mobile devices.

Thank YOU

What is SEG_Y? Headers and Traces.

SEG_Y is open standard file format for storing geophysical ( eg: seismic ) data. These are stored on magnetic tapes and usually of several Gigs in size.

• Headers :

contains optional SEG_Y tape label.
next 3200 bytes contains EBCDIC headers.
next 400 bytes contains Binary headers.

• Traces :

Traces contains Trace Header and Trace Data.
first 240 bytes contain Trace header.
next 4004 bytes contain Trace data.

SharePoint 2010 Products configuration wizard Errors & Fix

1. Exception – Failed to create the configuration database. An exception of type System.Security.Cryptography.CryptographicException was thrown. Additional exception information: The data is invalid.
Resolution – This has two steps
Step 1: Make sure that the “Network Service” account has full access to the “14” directory under %commonprogramfiles\Microsoft Shared\Web Server Extensions.
Step 2: Delete the registry key located under “SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\14.0\Secure\FarmAdmin” Registry key and then run the SharePoint 2010 Products Configuration Wizard.
It is likely that this registry key is required to be cleared each time you run the wizard after an unsuccessful attempt :)


2. Exception - Failed to register SharePoint Services. An exception of type System.Runtime.InteropServices.COMException was thrown. Additional exception information: Could not access the Search service configuration database.
I followed these steps and the configuration finished successfully.
1. On the Start menu, click Run. In the Open box, type regedit and then click OK.
2. In the Registry Editor, navigate to the following subkey, and then delete it:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\12.0\WSS\Services\Microsoft.SharePoint.Search.Administration.SPSearchService3.
 Run the SharePoint Products and Technologies Configuration Wizard again.

How to : working with HBase Delete API

• org.apache.hadoop.hbase.client.Delete

     HBase provide Delete to perform delete on a column(s), Column-Family(s) or entire Row, when Delete object is instantiated with a rowkey. 
     Delete accepts a Long Timestamp as parameter with Column-Family and a qualifier, which deletes all versions having smaller time-stamps. Delete creates a tombstone for any column or its version been deleted, HBase does the final deletion later when it goes for major compaction. 
IMPORTANT : If you try to 'put' data with the same timestamp which has been deleted recently, you'll not see it until HBase does its compaction. Though you'll not get any error or exception while doing  a 'put' but the same time you'll not see any result with 'scan' or 'get' until compaction happen. 
   If you don't provide a timestamp, default is current system time in milliseconds. 
Currently Update is not supported in HBase tables. A 'Delete' with 'put' is required to achieve this. If Update is on a column having multiple versions then timestamp plays critical role in maintaining the version order. Design your HBase schema accordingly :)

     To delete multiple rows or bulk delete, use 
public void delete(List<Delete> deletes)
            throws IOException method which is under HTable class.