Building, Running and Testing Apache Oozie 4.0.0
January 11, 2014Stream Data From Twitter To Analyze Using Hadoop
January 15, 2014Running out of memory is a common occurrence if you are using Amazon EC2 free tier instances to run Hadoop and other programs for your experiments. Follow the below steps to add Swap memory to your Amazon EC2 instance.
Log in to AWS management console and create (Elastic Block Store) EBS volume.
Select the size you need. For my needs, I have selected 1 GB. Make sure the zone you have selected is same as you instance’s zone to which you will be adding this memory.
Attach the newly created volume to the instance.
Login to your instance to which you have added the memory and run below commands to attach memory.
mkswap -f /dev/xvdf
swapon /dev/xvdf
Check the newly added Swap memory
watch -n 5 free -m
Edit your /etc/fstab file and add the below line so the memory automatically attach itself during reboot.
/dev/xvdf swap swap defaults 0 0
Big Data In Real World
We are a group of Big Data engineers who are passionate about Big Data and related Big Data technologies. We have designed, developed, deployed and maintained Big Data applications ranging from batch to real time streaming big data platforms. We have seen a wide range of real world big data problems, implemented some innovative and complex (or simple, depending on how you look at it) solutions.
1 Comment
[…] I was running on Amazon EC2 (free tier) so the process was failing because of no memory. So I had to add 1GB of virtual memory to the instance to proceed with the build. Follow this link for step by step instructions to add additional memory. […]