@InterfaceAudience.Private @InterfaceStability.Unstable public class DominantResourceFairnessPolicy extends SchedulingPolicy
| Modifier and Type | Class and Description |
|---|---|
static class |
DominantResourceFairnessPolicy.DominantResourceFairnessComparator
This class compares two
Schedulable instances according to the
DRF policy. |
| Modifier and Type | Field and Description |
|---|---|
static String |
NAME |
DEFAULT_POLICY| Constructor and Description |
|---|
DominantResourceFairnessPolicy() |
| Modifier and Type | Method and Description |
|---|---|
boolean |
checkIfUsageOverFairShare(org.apache.hadoop.yarn.api.records.Resource usage,
org.apache.hadoop.yarn.api.records.Resource fairShare)
Check if the resource usage is over the fair share under this policy.
|
void |
computeShares(Collection<? extends Schedulable> schedulables,
org.apache.hadoop.yarn.api.records.Resource totalResources)
Computes and updates the shares of
Schedulables as per
the SchedulingPolicy, to be used later for scheduling decisions. |
void |
computeSteadyShares(Collection<? extends FSQueue> queues,
org.apache.hadoop.yarn.api.records.Resource totalResources)
Computes and updates the steady shares of
FSQueues as per the
SchedulingPolicy. |
Comparator<Schedulable> |
getComparator()
The comparator returned by this method is to be used for sorting the
Schedulables in that queue. |
org.apache.hadoop.yarn.api.records.Resource |
getHeadroom(org.apache.hadoop.yarn.api.records.Resource queueFairShare,
org.apache.hadoop.yarn.api.records.Resource queueUsage,
org.apache.hadoop.yarn.api.records.Resource maxAvailable)
Get headroom by calculating the min of
clusterAvailable and
(queueFairShare - queueUsage) resources that are
applicable to this policy. |
String |
getName() |
org.apache.hadoop.yarn.util.resource.ResourceCalculator |
getResourceCalculator()
The
ResourceCalculator returned by this method should be used
for any calculations involving resources. |
void |
initialize(FSContext fsContext)
Initialize the scheduling policy with a
FSContext object, which has
a pointer to the cluster resources among other information. |
getInstance, initialize, isChildPolicyAllowed, parsepublic static final String NAME
public String getName()
getName in class SchedulingPolicySchedulingPolicypublic Comparator<Schedulable> getComparator()
SchedulingPolicySchedulables in that queue.getComparator in class SchedulingPolicypublic org.apache.hadoop.yarn.util.resource.ResourceCalculator getResourceCalculator()
SchedulingPolicyResourceCalculator returned by this method should be used
for any calculations involving resources.getResourceCalculator in class SchedulingPolicypublic void computeShares(Collection<? extends Schedulable> schedulables, org.apache.hadoop.yarn.api.records.Resource totalResources)
SchedulingPolicySchedulables as per
the SchedulingPolicy, to be used later for scheduling decisions.
The shares computed are instantaneous and only consider queues with
running applications.computeShares in class SchedulingPolicyschedulables - Schedulables whose shares are to be updatedtotalResources - Total Resources in the clusterpublic void computeSteadyShares(Collection<? extends FSQueue> queues, org.apache.hadoop.yarn.api.records.Resource totalResources)
SchedulingPolicyFSQueues as per the
SchedulingPolicy. The steady share does not differentiate
between queues with and without running applications under them. The
steady share is not used for scheduling, it is displayed on the Web UI
for better visibility.computeSteadyShares in class SchedulingPolicyqueues - FSQueues whose shares are to be updatedtotalResources - Total Resources in the clusterpublic boolean checkIfUsageOverFairShare(org.apache.hadoop.yarn.api.records.Resource usage,
org.apache.hadoop.yarn.api.records.Resource fairShare)
SchedulingPolicycheckIfUsageOverFairShare in class SchedulingPolicyusage - Resource the resource usagefairShare - Resource the fair sharepublic org.apache.hadoop.yarn.api.records.Resource getHeadroom(org.apache.hadoop.yarn.api.records.Resource queueFairShare,
org.apache.hadoop.yarn.api.records.Resource queueUsage,
org.apache.hadoop.yarn.api.records.Resource maxAvailable)
SchedulingPolicyclusterAvailable and
(queueFairShare - queueUsage) resources that are
applicable to this policy. For eg if only memory then leave other
resources such as CPU to same as clusterAvailable.getHeadroom in class SchedulingPolicyqueueFairShare - fairshare in the queuequeueUsage - resources used in the queuemaxAvailable - available resource in cluster for this queuepublic void initialize(FSContext fsContext)
SchedulingPolicyFSContext object, which has
a pointer to the cluster resources among other information.initialize in class SchedulingPolicyfsContext - a FSContext object which has a pointer to the
cluster resourcesCopyright © 2008–2022 Apache Software Foundation. All rights reserved.