Untitled

Big data

Hadoop / HDFS (storage)
MR (Mappper Reducer) (Process the data into hadoop)
YARN (Yet another resource negogiator) (Resource Manager increase data processing speed)

Core component

Hive
Sqoop
Spark

Hdfs/ Hadoop

Demoans > Background running servers...

Namenode >> Master node which control the cluster..It contains metadata about the data store in data node
Data Node >> Actual Data will be stored.
Secondary name node
Backup / stand by name node..

emp
eid ename esal
1,raj,1000
2,Ram,2000


each block 128mb
non sequentialy block store in dn
each block replicate 3 times (default replication factor is 3)

each and every 3 sec data node send heart beat signal to name node
each and every 30 sec data node send blocks information to name node
name node will wait 180 sec for data node signal..resilieant time (180 Sec)

MR

Job tracker  (Master Deamons)
Task tracker (Slave )

YARN

Resource Manager 
================
It is master deamon one per cluster,
It handle the task, 
It collect all resources information from the node managers and release cotainer to perform the task in each node.

Node Manager
=============
It send the resource infortmation to resource manager
and it monitor the container resource utlization.

If container exceed resource which alocatted by resource manager 
node manager kill the container.

Application Master
==================
It is a Short term deamon whenever the container released application master
come to live and it monitor container Task status once container task completed
all containers report to application master
AM send report Resource manager.