Sample Image

Description

A Hadoop Job can be in several states PREP, RUNNING, COMPLETE, KILLED, or FAILED. PREP and RUNNING are temporary states, while COMPLETE, KILLED ,and FAILED are final states. The fetch code here connects to the JobTracker and builds a list of jobs for the last 5 minutes. We then total each state and report that. This graph is very useful in that it shows at a given time how many Job are running.

Sample Output

            bash-3.2# sh JobClient.sh sample.xml
            mapTasks:0 maxMapTasks:2 reduceTasks:0 maxReduceTasks:2 \
            jobsToCompleteSize:0 PREP:1 RUNNING:2 SUCCEEDED:0 FAILED:3 KILLED:0
          

Code Specific

Behind the scenes the JobTracker stores this information. The JobClient class, the one use use to submit jobs to Hadoop, is used to access this information. The fetch code uses reflection to deal with changes in the JobClient API. This also allows us to have one build work against all versions.

Continue to the Maps vs Reduces Graph