Cloudera Quick Start VM Big Data Assignment Help | Implement Big Data Map Reduce Problem

Problem Requirement

Implementation

Data:

mapper.py

#!/usr/bin/env python
  
# import sys because we need to read and write data to STDIN and STDOUT
import sys
  
# reading entire line from STDIN (standard input)
for line in sys.stdin:
    # to remove leading and trailing whitespace
    line = line.strip()
    # split the line into words
    words = line.split()
      
    # we are looping over the words array and printing the word
    # with the count of 1 to the STDOUT
    # for word in words:
        # write the results to STDOUT (standard output);
        # what we output here will be the input for the
        # Reduce step, i.e. the input for reducer.py
    print ('%s\t%s' % (words[0], words[1]))

reducer.py

#!/usr/bin/env python

from operator import itemgetter

import sys

###################################

Write code here

##################################

How to run it

We make a directory in Hadoop named input in the location /user/cloudera/input as can be seen above, here we will be transferring our data.txt file

Here we transfer the data.txt in directory we just created (input directory)

In second command we check list of files in the directory /user/cloudera/input and we find that our data.txt file is successfully moved there

Using the -cat command we see the contants of data.txt files which contains each English word 3 times and its corresponding translation in 3 different languages

Now its time to run our mapper.py and reducer.py files on the input data.txt file we enter the command and wait for the completion of execution

Screenshot part where the end represents the progress has started and currently execution of mapper.py or map utility is 0% and

reducer.py or reduce is also 0%

Here at 13:49:31 we see that map utility is done 100% and at 13:52:13 we see that reduce utility is also completed 100% and therefore completing our task

Finally we see the result of our output file and it gives English letters and their corresponding translations in the same line.

Also below we see the execution of the files done locally without Hadoop:-

Hire expert to get help in any Big Data related assignment help, project help or homework help.

Realcode4you.com expert team provide code without any plagiarism issue with an affordable price.

Send your project requirement details at:

realcode4you@gmail.com