public class DefaultContainerExecutor extends ContainerExecutor
DefaultContainerExecuter class offers generic container
execution services. Process execution is handled in a platform-independent
way via ProcessBuilder.| Modifier and Type | Class and Description |
|---|---|
protected class |
DefaultContainerExecutor.LocalWrapperScriptBuilder
This class is a utility to create a wrapper script that is platform
appropriate.
|
ContainerExecutor.DelayedProcessKiller, ContainerExecutor.ExitCode, ContainerExecutor.Signal| Modifier and Type | Field and Description |
|---|---|
protected org.apache.hadoop.fs.FileContext |
lfs
A
FileContext for the local file system. |
DIRECTORY_CONTENTS, TASK_LAUNCH_SCRIPT_PERMISSION, TOKEN_FILE_NAME_FMT, WILDCARD| Constructor and Description |
|---|
DefaultContainerExecutor()
Default constructor for use in testing.
|
| Modifier and Type | Method and Description |
|---|---|
protected org.apache.hadoop.util.Shell.CommandExecutor |
buildCommandExecutor(String wrapperScriptPath,
String containerIdStr,
String user,
org.apache.hadoop.fs.Path pidFile,
org.apache.hadoop.yarn.api.records.Resource resource,
File workDir,
Map<String,String> environment)
Create a new
Shell.ShellCommandExecutor using the parameters. |
void |
clearLogDirPermissions()
Clear the internal variable for repeatable testing.
|
static boolean |
containerIsAlive(String pid)
Returns true if the process with the specified pid is alive.
|
protected void |
copyFile(org.apache.hadoop.fs.Path src,
org.apache.hadoop.fs.Path dst,
String owner)
Copy a file using the
lfs FileContext. |
protected ContainerLocalizer |
createContainerLocalizer(String user,
String appId,
String locId,
String tokenFileName,
List<String> localDirs,
org.apache.hadoop.fs.FileContext localizerFc)
Create a new
ContainerLocalizer instance. |
protected void |
createDir(org.apache.hadoop.fs.Path dirPath,
org.apache.hadoop.fs.permission.FsPermission perms,
boolean createParent,
String user)
Use the
lfs FileContext to create the target directory. |
void |
deleteAsUser(DeletionAsUserContext ctx)
Delete specified directories as a given user.
|
protected DefaultContainerExecutor.LocalWrapperScriptBuilder |
getLocalWrapperScriptBuilder(String containerIdStr,
org.apache.hadoop.fs.Path containerWorkDir)
Create a
DefaultContainerExecutor.LocalWrapperScriptBuilder for the given container ID
and path that is appropriate to the current platform. |
String |
getLogDirPermissions()
Return the default container log directory permissions.
|
protected org.apache.hadoop.fs.Path |
getWorkingDir(List<String> localDirs,
String user,
String appId)
Return a randomly chosen application directory from a list of local storage
directories.
|
void |
init(Context nmContext)
Run the executor initialization steps.
|
boolean |
isContainerAlive(ContainerLivenessContext ctx)
Check if a container is alive.
|
protected void |
killContainer(String pid,
ContainerExecutor.Signal signal)
Send a specified signal to the specified pid
|
int |
launchContainer(ContainerStartContext ctx)
Launch the container on the node.
|
protected void |
setScriptExecutable(org.apache.hadoop.fs.Path script,
String owner)
Make a file executable using the
lfs FileContext. |
boolean |
signalContainer(ContainerSignalContext ctx)
Signal container with the specified signal.
|
void |
startLocalizer(LocalizerStartContext ctx)
Prepare the environment for containers in this application to execute.
|
void |
symLink(String target,
String symlink)
Create a symlink file which points to the target.
|
activateContainer, deactivateContainer, getConf, getIpAndHost, getLocalIpAndHost, getNMEnvVar, getPidFilePath, getProcessId, getRunCommand, getRunCommand, getRunCommandForOther, getRunCommandForWindows, isContainerActive, localizeClasspathJar, logOutput, pauseContainer, reacquireContainer, readDirAsUser, resumeContainer, setConf, writeLaunchEnv, writeLaunchEnvprotected final org.apache.hadoop.fs.FileContext lfs
FileContext for the local file system.public DefaultContainerExecutor()
protected void copyFile(org.apache.hadoop.fs.Path src,
org.apache.hadoop.fs.Path dst,
String owner)
throws IOException
lfs FileContext.src - the file to copydst - where to copy the fileowner - the owner of the new copy. Used only in secure Windows
clustersIOException - when the copy failsWindowsSecureContainerExecutorprotected void setScriptExecutable(org.apache.hadoop.fs.Path script,
String owner)
throws IOException
lfs FileContext.script - the path to make executableowner - the new owner for the file. Used only in secure Windows
clustersIOException - when the change mode operation failsWindowsSecureContainerExecutorpublic void init(Context nmContext) throws IOException
ContainerExecutorinit in class ContainerExecutornmContext - Context of NMIOException - if initialization failspublic void startLocalizer(LocalizerStartContext ctx) throws IOException, InterruptedException
ContainerExecutorFor $x in local.dirs create $x/$user/$appId Copy $nmLocal/appTokens -> $N/$user/$appId For $rsrc in private resources Copy $rsrc -> $N/$user/filecache/[idef] For $rsrc in job resources Copy $rsrc -> $N/$user/$appId/filecache/idef
startLocalizer in class ContainerExecutorctx - LocalizerStartContext that encapsulates necessary information
for starting a localizer.IOException - for most application init failuresInterruptedException - if application init thread is halted by NM@InterfaceAudience.Private protected ContainerLocalizer createContainerLocalizer(String user, String appId, String locId, String tokenFileName, List<String> localDirs, org.apache.hadoop.fs.FileContext localizerFc) throws IOException
ContainerLocalizer instance.user - the user who owns the job for which the localization is being
runappId - the ID of the application for which the localization is being
runlocId - the ID of the container for which the localization is being
runlocalDirs - a list of directories to use as destinations for the
localizationlocalizerFc - the FileContext to use when localizing filesContainerLocalizer instanceIOException - if user or locId is null or if
the container localizer has an initialization failurepublic int launchContainer(ContainerStartContext ctx) throws IOException, org.apache.hadoop.yarn.exceptions.ConfigurationException
ContainerExecutorlaunchContainer in class ContainerExecutorctx - Encapsulates information necessary for launching containers.IOException - if the container launch failsorg.apache.hadoop.yarn.exceptions.ConfigurationException - if config error was foundprotected org.apache.hadoop.util.Shell.CommandExecutor buildCommandExecutor(String wrapperScriptPath, String containerIdStr, String user, org.apache.hadoop.fs.Path pidFile, org.apache.hadoop.yarn.api.records.Resource resource, File workDir, Map<String,String> environment)
Shell.ShellCommandExecutor using the parameters.wrapperScriptPath - the path to the script to executecontainerIdStr - the container IDuser - the application owner's usernamepidFile - the path to the container's PID fileresource - this parameter controls memory and CPU limits.workDir - If not-null, specifies the directory which should be set
as the current working directory for the command. If null,
the current working directory is not modified.environment - the container environmentShell.ShellCommandExecutorShell.ShellCommandExecutorprotected DefaultContainerExecutor.LocalWrapperScriptBuilder getLocalWrapperScriptBuilder(String containerIdStr, org.apache.hadoop.fs.Path containerWorkDir)
DefaultContainerExecutor.LocalWrapperScriptBuilder for the given container ID
and path that is appropriate to the current platform.containerIdStr - the container IDcontainerWorkDir - the container's working directoryDefaultContainerExecutor.LocalWrapperScriptBuilderpublic boolean signalContainer(ContainerSignalContext ctx) throws IOException
ContainerExecutorsignalContainer in class ContainerExecutorctx - Encapsulates information necessary for signaling containers.IOException - if signaling the container failspublic boolean isContainerAlive(ContainerLivenessContext ctx) throws IOException
ContainerExecutorisContainerAlive in class ContainerExecutorctx - Encapsulates information necessary for container liveness check.IOException - if there is a failure while checking the container
statuspublic static boolean containerIsAlive(String pid) throws IOException
pid - String pidIOException - if the command to test process liveliness failsprotected void killContainer(String pid, ContainerExecutor.Signal signal) throws IOException
pid - the pid of the process [group] to signal.signal - signal to sendIOException - if the command to kill the process failspublic void deleteAsUser(DeletionAsUserContext ctx) throws IOException, InterruptedException
ContainerExecutordeleteAsUser in class ContainerExecutorctx - Encapsulates information necessary for deletion.IOException - if delete failsInterruptedException - if interrupted while waiting for the deletion
operation to completepublic void symLink(String target, String symlink) throws IOException
ContainerExecutorsymLink in class ContainerExecutortarget - The target for symlinksymlink - the symlink fileIOException - Error when creating symlinksprotected org.apache.hadoop.fs.Path getWorkingDir(List<String> localDirs, String user, String appId) throws IOException
localDirs - the target directories from which to selectuser - the user who owns the applicationappId - the application IDIOException - if no application directories for the user can be
foundprotected void createDir(org.apache.hadoop.fs.Path dirPath,
org.apache.hadoop.fs.permission.FsPermission perms,
boolean createParent,
String user)
throws IOException
lfs FileContext to create the target directory.dirPath - the target directoryperms - the target permissions for the target directorycreateParent - whether the parent directories should also be createduser - the user as whom the target directory should be created.
Used only on secure Windows hosts.IOException - if there's a failure performing a file operationWindowsSecureContainerExecutorpublic String getLogDirPermissions()
public void clearLogDirPermissions()
Copyright © 2022 Apache Software Foundation. All rights reserved.