public class ExponentiallySmoothedTaskRuntimeEstimator extends Object
| Modifier and Type | Class and Description |
|---|---|
static class |
ExponentiallySmoothedTaskRuntimeEstimator.SmoothedValue |
| Modifier and Type | Field and Description |
|---|---|
protected AppContext |
context |
protected Set<Task> |
doneTasks |
protected Map<Job,DataStatistics> |
mapperStatistics |
protected Map<Job,DataStatistics> |
reducerStatistics |
protected Map<org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId,Long> |
startTimes |
| Constructor and Description |
|---|
ExponentiallySmoothedTaskRuntimeEstimator() |
| Modifier and Type | Method and Description |
|---|---|
long |
attemptEnrolledTime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID) |
void |
contextualize(org.apache.hadoop.conf.Configuration conf,
AppContext context) |
protected DataStatistics |
dataStatisticsForTask(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID) |
void |
enrollAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status,
long timestamp) |
long |
estimatedNewAttemptRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId id)
Estimates how long a new attempt on this task will take if we start
one now
|
long |
estimatedRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId id)
Estimate a task attempt's total runtime.
|
boolean |
hasStagnatedProgress(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId id,
long timeStamp)
Returns true if the estimator has no updates records for a threshold time
window.
|
long |
runtimeEstimateVariance(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId id)
Computes the width of the error band of our estimate of the task
runtime as returned by
TaskRuntimeEstimator.estimatedRuntime(TaskAttemptId) |
long |
thresholdRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID)
Find a maximum reasonable execution wallclock time.
|
void |
updateAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status,
long timestamp) |
protected AppContext context
protected final Map<org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId,Long> startTimes
protected final Map<Job,DataStatistics> mapperStatistics
protected final Map<Job,DataStatistics> reducerStatistics
public ExponentiallySmoothedTaskRuntimeEstimator()
public void contextualize(org.apache.hadoop.conf.Configuration conf,
AppContext context)
contextualize in interface TaskRuntimeEstimatorpublic long estimatedRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId id)
TaskRuntimeEstimatorid - the TaskAttemptId of the attempt we are asking about-1 if
we don't have enough information yet to produce an estimate.public long runtimeEstimateVariance(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId id)
TaskRuntimeEstimatorTaskRuntimeEstimator.estimatedRuntime(TaskAttemptId)id - the TaskAttemptId of the attempt we are asking about-1 if
we don't have enough information yet to produce an estimate.public void updateAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status, long timestamp)
updateAttempt in interface TaskRuntimeEstimatorpublic void enrollAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status, long timestamp)
enrollAttempt in interface TaskRuntimeEstimatorpublic long attemptEnrolledTime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)
attemptEnrolledTime in interface TaskRuntimeEstimatorprotected DataStatistics dataStatisticsForTask(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID)
public long thresholdRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID)
TaskRuntimeEstimatorthresholdRuntime in interface TaskRuntimeEstimatortaskID - the TaskId of the task we are asking aboutpublic long estimatedNewAttemptRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId id)
TaskRuntimeEstimatorestimatedNewAttemptRuntime in interface TaskRuntimeEstimatorid - the TaskId of the task we are asking about-1 if
we don't have enough information yet to produce an estimate.public boolean hasStagnatedProgress(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId id,
long timeStamp)
TaskRuntimeEstimatorhasStagnatedProgress in interface TaskRuntimeEstimatorid - the TaskAttemptId of the attempt we are asking abouttimeStamp - the time of the report we compare withCopyright © 2022 Apache Software Foundation. All rights reserved.