Difference between revisions of "Apache Hadoop"

From wikieduonline
Jump to navigation Jump to search
 
(3 intermediate revisions by the same user not shown)
Line 4: Line 4:
 
* [[MapReduce]]
 
* [[MapReduce]]
 
* <code>[[brew install hadoop]]</code>
 
* <code>[[brew install hadoop]]</code>
 +
[[hadoop fs]]
  
  
−
There are currently several monitoring platforms to track HDFS performance, including [[Hortonworks]], [[Cloudera]], and Datadog.
+
There are currently several monitoring platforms to track HDFS performance, including [[Hortonworks]], [[Cloudera]], and [[Datadog]].
 +
 
 +
== Example ==
 +
<pre>
 +
hadoop
 +
Usage: hadoop [OPTIONS] SUBCOMMAND [SUBCOMMAND OPTIONS]
 +
or    hadoop [OPTIONS] CLASSNAME [CLASSNAME OPTIONS]
 +
  where CLASSNAME is a user-provided Java class
 +
 
 +
  OPTIONS is none or any of:
 +
 
 +
--config dir                    Hadoop config directory
 +
--debug                          turn on shell script debug mode
 +
--help                          usage information
 +
buildpaths                      attempt to add class files from
 +
                                build tree
 +
hostnames list[,of,host,names]  hosts to use in worker mode
 +
hosts filename                  list of hosts to use in worker
 +
                                mode
 +
loglevel level                  set the log4j level for this
 +
                                command
 +
workers                          turn on worker mode
 +
 
 +
  SUBCOMMAND is one of:
 +
 
 +
 
 +
    Admin Commands:
 +
 
 +
daemonlog    get/set the log level for each daemon
 +
 
 +
    Client Commands:
 +
 
 +
archive      create a Hadoop archive
 +
checknative  check native Hadoop and compression libraries
 +
              availability
 +
classpath    prints the class path needed to get the Hadoop jar
 +
              and the required libraries
 +
conftest      validate configuration XML files
 +
credential    interact with credential providers
 +
distch        distributed metadata changer
 +
distcp        copy file or directories recursively
 +
dtutil        operations related to delegation tokens
 +
envvars      display computed Hadoop environment variables
 +
fedbalance    balance data between sub-clusters
 +
fs            run a generic filesystem user client
 +
gridmix      submit a mix of synthetic job, modeling a profiled
 +
              from production load
 +
jar <jar>    run a jar file. NOTE: please use "yarn jar" to
 +
              launch YARN applications, not this command.
 +
jnipath      prints the java.library.path
 +
kdiag        Diagnose Kerberos Problems
 +
kerbname      show auth_to_local principal conversion
 +
key          manage keys via the KeyProvider
 +
rbfbalance    move directories and files across router-based
 +
              federation namespaces
 +
rumenfolder  scale a rumen input trace
 +
rumentrace    convert logs into a rumen trace
 +
s3guard      S3 Commands
 +
version      print the version
 +
 
 +
    Daemon Commands:
 +
 
 +
kms          run KMS, the Key Management Server
 +
registrydns  run the registry DNS server
 +
 
 +
SUBCOMMAND may print help when invoked w/o parameters or with -h.
 +
</pre>
 +
 
  
  
Line 17: Line 85:
 
* {{Cloudera}}
 
* {{Cloudera}}
 
* {{Apache}}
 
* {{Apache}}
−
 
  
 
[[Category:Computing]]
 
[[Category:Computing]]
 +
[[Category:Apache]]

Latest revision as of 18:29, 9 April 2024

wikipedia:Hadoop (2006) 

hadoop fs


There are currently several monitoring platforms to track HDFS performance, including Hortonworks, Cloudera, and Datadog.

Example[edit]

hadoop
Usage: hadoop [OPTIONS] SUBCOMMAND [SUBCOMMAND OPTIONS]
 or    hadoop [OPTIONS] CLASSNAME [CLASSNAME OPTIONS]
  where CLASSNAME is a user-provided Java class

  OPTIONS is none or any of:

--config dir                     Hadoop config directory
--debug                          turn on shell script debug mode
--help                           usage information
buildpaths                       attempt to add class files from
                                 build tree
hostnames list[,of,host,names]   hosts to use in worker mode
hosts filename                   list of hosts to use in worker
                                 mode
loglevel level                   set the log4j level for this
                                 command
workers                          turn on worker mode

  SUBCOMMAND is one of:


    Admin Commands:

daemonlog     get/set the log level for each daemon

    Client Commands:

archive       create a Hadoop archive
checknative   check native Hadoop and compression libraries
              availability
classpath     prints the class path needed to get the Hadoop jar
              and the required libraries
conftest      validate configuration XML files
credential    interact with credential providers
distch        distributed metadata changer
distcp        copy file or directories recursively
dtutil        operations related to delegation tokens
envvars       display computed Hadoop environment variables
fedbalance    balance data between sub-clusters
fs            run a generic filesystem user client
gridmix       submit a mix of synthetic job, modeling a profiled
              from production load
jar <jar>     run a jar file. NOTE: please use "yarn jar" to
              launch YARN applications, not this command.
jnipath       prints the java.library.path
kdiag         Diagnose Kerberos Problems
kerbname      show auth_to_local principal conversion
key           manage keys via the KeyProvider
rbfbalance    move directories and files across router-based
              federation namespaces
rumenfolder   scale a rumen input trace
rumentrace    convert logs into a rumen trace
s3guard       S3 Commands
version       print the version

    Daemon Commands:

kms           run KMS, the Key Management Server
registrydns   run the registry DNS server

SUBCOMMAND may print help when invoked w/o parameters or with -h.


Related terms[edit]

See also[edit]

Retrieved from "https://www.wikieduonline.com/index.php?title=Apache_Hadoop&oldid=312985"

Advertising: