@InterfaceAudience.Private @InterfaceStability.Unstable public class LogAggregationIndexedFileController extends LogAggregationFileController
| Modifier and Type | Class and Description |
|---|---|
static class |
LogAggregationIndexedFileController.IndexedFileLogMeta
This IndexedFileLogMeta includes the meta information
for a single file which would be aggregated in one
Log aggregation cycle.
|
static class |
LogAggregationIndexedFileController.IndexedLogsMeta
This IndexedLogsMeta includes all the meta information
for the aggregated log file.
|
static class |
LogAggregationIndexedFileController.IndexedPerAggregationLogMeta
This IndexedPerAggregationLogMeta includes the meta information
for all files which would be aggregated in one
Log aggregation cycle.
|
| Modifier and Type | Field and Description |
|---|---|
static String |
CHECK_SUM_FILE_SUFFIX |
APP_DIR_PERMISSIONS, APP_LOG_FILE_UMASK, conf, fileControllerName, fsSupportsChmod, LOG_AGGREGATION_FS_SUPPORT_APPEND, remoteRootLogDir, remoteRootLogDirSuffix, retentionSize, TLDIR_PERMISSIONS| Constructor and Description |
|---|
LogAggregationIndexedFileController() |
| Modifier and Type | Method and Description |
|---|---|
void |
closeWriter()
Close the writer.
|
org.apache.hadoop.fs.FileStatus |
getAllChecksumFiles(Map<String,org.apache.hadoop.fs.FileStatus> fileMap,
String fileName) |
Map<org.apache.hadoop.yarn.api.records.ApplicationAccessType,String> |
getApplicationAcls(org.apache.hadoop.fs.Path aggregatedLogPath,
org.apache.hadoop.yarn.api.records.ApplicationId appId)
Returns ACLs for the application.
|
String |
getApplicationOwner(org.apache.hadoop.fs.Path aggregatedLogPath,
org.apache.hadoop.yarn.api.records.ApplicationId appId)
Returns the owner of the application.
|
static int |
getFSInputBufferSize(org.apache.hadoop.conf.Configuration conf) |
static int |
getFSOutputBufferSize(org.apache.hadoop.conf.Configuration conf) |
List<org.apache.hadoop.fs.FileStatus> |
getNodeLogFileToRead(List<org.apache.hadoop.fs.FileStatus> nodeFiles,
String nodeId,
org.apache.hadoop.yarn.api.records.ApplicationId appId) |
org.apache.hadoop.fs.Path |
getRemoteAppLogDir(org.apache.hadoop.yarn.api.records.ApplicationId appId,
String user)
Get the remote application directory for log aggregation.
|
long |
getRollOverLogMaxSize(org.apache.hadoop.conf.Configuration conf) |
Clock |
getSystemClock() |
void |
initializeWriter(LogAggregationFileControllerContext context)
Initialize the writer.
|
void |
initInternal(org.apache.hadoop.conf.Configuration conf)
Derived classes initialize themselves using this method.
|
boolean |
isRollover(org.apache.hadoop.fs.FileContext fc,
org.apache.hadoop.fs.Path candidate) |
LogAggregationIndexedFileController.IndexedLogsMeta |
loadIndexedLogsMeta(org.apache.hadoop.fs.Path remoteLogPath,
long end,
org.apache.hadoop.yarn.api.records.ApplicationId appId) |
Map<String,Long> |
parseCheckSumFiles(List<org.apache.hadoop.fs.FileStatus> fileList) |
void |
postWrite(LogAggregationFileControllerContext record)
Operations needed after write the log content.
|
boolean |
readAggregatedLogs(ContainerLogsRequest logRequest,
OutputStream os)
Output container log.
|
List<ContainerLogMeta> |
readAggregatedLogsMeta(ContainerLogsRequest logRequest)
Return a list of
ContainerLogMeta for an application
from Remote FileSystem. |
void |
renderAggregatedLogsBlock(HtmlBlock.Block html,
View.ViewContext context)
Render Aggregated Logs block.
|
void |
write(AggregatedLogFormat.LogKey logKey,
AggregatedLogFormat.LogValue logValue)
Write the log content.
|
aggregatedLogSuffix, belongsToAppAttempt, checkExists, cleanOldLogs, closePrintStream, createAppDir, createDir, getFileSystem, getRemoteNodeLogFileForApp, getRemoteRootLogDir, getRemoteRootLogDirSuffix, initialize, isFsSupportsChmod, verifyAndCreateRemoteLogDirpublic static final String CHECK_SUM_FILE_SUFFIX
public void initInternal(org.apache.hadoop.conf.Configuration conf)
LogAggregationFileControllerinitInternal in class LogAggregationFileControllerconf - the Configurationpublic void initializeWriter(LogAggregationFileControllerContext context) throws IOException
LogAggregationFileControllerinitializeWriter in class LogAggregationFileControllercontext - the LogAggregationFileControllerContextIOException - if fails to initialize the writerpublic void closeWriter()
LogAggregationFileControllercloseWriter in class LogAggregationFileControllerpublic void write(AggregatedLogFormat.LogKey logKey, AggregatedLogFormat.LogValue logValue) throws IOException
LogAggregationFileControllerwrite in class LogAggregationFileControllerlogKey - the log keylogValue - the log contentIOException - if fails to write the logspublic void postWrite(LogAggregationFileControllerContext record) throws Exception
LogAggregationFileControllerpostWrite in class LogAggregationFileControllerrecord - the LogAggregationFileControllerContextException - if anything failspublic boolean readAggregatedLogs(ContainerLogsRequest logRequest, OutputStream os) throws IOException
LogAggregationFileControllerreadAggregatedLogs in class LogAggregationFileControllerlogRequest - ContainerLogsRequestos - the output streamIOException - if we can not access the log file.public List<ContainerLogMeta> readAggregatedLogsMeta(ContainerLogsRequest logRequest) throws IOException
LogAggregationFileControllerContainerLogMeta for an application
from Remote FileSystem.readAggregatedLogsMeta in class LogAggregationFileControllerlogRequest - ContainerLogsRequestContainerLogMetaIOException - if there is no available log file@InterfaceAudience.Private public Map<String,Long> parseCheckSumFiles(List<org.apache.hadoop.fs.FileStatus> fileList) throws IOException
IOException@InterfaceAudience.Private public List<org.apache.hadoop.fs.FileStatus> getNodeLogFileToRead(List<org.apache.hadoop.fs.FileStatus> nodeFiles, String nodeId, org.apache.hadoop.yarn.api.records.ApplicationId appId) throws IOException
IOException@InterfaceAudience.Private public org.apache.hadoop.fs.FileStatus getAllChecksumFiles(Map<String,org.apache.hadoop.fs.FileStatus> fileMap, String fileName)
public void renderAggregatedLogsBlock(HtmlBlock.Block html, View.ViewContext context)
LogAggregationFileControllerrenderAggregatedLogsBlock in class LogAggregationFileControllerhtml - the htmlcontext - the ViewContextpublic String getApplicationOwner(org.apache.hadoop.fs.Path aggregatedLogPath, org.apache.hadoop.yarn.api.records.ApplicationId appId) throws IOException
LogAggregationFileControllergetApplicationOwner in class LogAggregationFileControlleraggregatedLogPath - the aggregatedLog pathappId - the ApplicationIdIOException - if we can not get the application ownerpublic Map<org.apache.hadoop.yarn.api.records.ApplicationAccessType,String> getApplicationAcls(org.apache.hadoop.fs.Path aggregatedLogPath, org.apache.hadoop.yarn.api.records.ApplicationId appId) throws IOException
LogAggregationFileControllergetApplicationAcls in class LogAggregationFileControlleraggregatedLogPath - the aggregatedLog path.appId - the ApplicationIdIOException - if we can not get the application aclspublic org.apache.hadoop.fs.Path getRemoteAppLogDir(org.apache.hadoop.yarn.api.records.ApplicationId appId,
String user)
throws IOException
LogAggregationFileControllergetRemoteAppLogDir in class LogAggregationFileControllerappId - the Application IDuser - the Application OwnerIOException - if can not find the remote application directory@InterfaceAudience.Private public LogAggregationIndexedFileController.IndexedLogsMeta loadIndexedLogsMeta(org.apache.hadoop.fs.Path remoteLogPath, long end, org.apache.hadoop.yarn.api.records.ApplicationId appId) throws IOException
IOException@InterfaceAudience.Private public static int getFSOutputBufferSize(org.apache.hadoop.conf.Configuration conf)
@InterfaceAudience.Private public static int getFSInputBufferSize(org.apache.hadoop.conf.Configuration conf)
@InterfaceAudience.Private public long getRollOverLogMaxSize(org.apache.hadoop.conf.Configuration conf)
@InterfaceAudience.Private
public boolean isRollover(org.apache.hadoop.fs.FileContext fc,
org.apache.hadoop.fs.Path candidate)
throws IOException
IOException@InterfaceAudience.Private public Clock getSystemClock()
Copyright © 2008–2022 Apache Software Foundation. All rights reserved.