Hadoop installation notes

Cluster planning:

• Small cluster (2-10 nodes): Clusters of three or more machines typically use a dedicated NameNode/ResourceManager, and all other nodes are workers. 
• Medium cluster (10-40 nodes): separate machines for master, usually dedicated edge node 
• Large cluster (> 40 nodes): occupies multiple racks, requires individual planning

Storage layer for HDFS:

• Cloud and virtualization solutions are very popular in the enterprise world 
• With Hadoop, focus on using JDOD (just bunch of disks) instead of SAN (storage area network) 
• Hadoop heterogeneous storage was introduced in Hadoop 2.5

Commodity hardware:

Balanced configuration for one worker recommended by vendors:

• 2-4 8-cores CPU 
• 128 GB RAM 
• 12 hard drives (1 or 2 TB each)