Sample Image

Description

Given that a Hadoop system has a finite number of maximum Mappers and Reduces that can be running at once, knowing how many Map tasks or reduce Tasks Running at a given time can aid in job sceduling and help understand how the cluster is being utilized.

Sample Output

            bash-3.2# sh JobClient.sh sample.xml
            mapTasks:0 maxMapTasks:2 reduceTasks:0 maxReduceTasks:2 \
            jobsToCompleteSize:0 PREP:1 RUNNING:2 SUCCEEDED:0 FAILED:3 KILLED:0
          

Code Specific

Behind the scenes the JobTracker stores this information. The JobClient class, the one use use to submit jobs to Hadoop is used to access this information. Our fetch code uses reflection to deal with changes in the JobClient API. This also allows us to have one build work against all versions.

Continue to Files Total