redhat7源码编译hadoop2.6.0

    Windows Windows 1年前 (2019-08-13) 65次浏览 已收录 0个评论 扫描二维码

    redhat7源码编译hadoop2.6.0。以前在32位linux机器上编译过hadoop2.6.0,这次在redhat7 64bit上再次编译hadoop2.6.0,除必须的jdk,maven,protobuf需要安装之外,还需要安装系统依赖库gcc,gcc-c++,ncurses-devel,openssl-devel,cmake,这些依赖库的安装可以直接通过yum命令一下安装: yum install -y gcc gcc-c++ cmake openssl-devel ncurses-devel,依赖库之间通过空格分割。

    这次的安装还是遇到了只能使用jdk1.7编译的问题。如果使用jdk1.8,会报如下图所示的错误。[ERROR] Exit code:1 – /home/hadoop/hadoop-2.6.0-src/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/InterfaceStability.java:27: error: unexcepted end tag:

    redhat7源码编译hadoop2.6.0

    遇到这个错误,我们只能将jdk换成1.7,之后顺利编译通过。

    编译命令:mvn package -DskipTests -Pdist,native -Dtar

    redhat7源码编译hadoop2.6.0

    build success

    INFO] ————————————————————————

    [INFO] Reactor Summary:

    [INFO]

    [INFO] Apache Hadoop Main …………………………… SUCCESS [ 1.041 s]

    [INFO] Apache Hadoop Project POM …………………….. SUCCESS [ 1.018 s]

    [INFO] Apache Hadoop Annotations …………………….. SUCCESS [ 1.937 s]

    [INFO] Apache Hadoop Assemblies ……………………… SUCCESS [ 0.255 s]

    [INFO] Apache Hadoop Project Dist POM ………………… SUCCESS [ 15.730 s]

    [INFO] Apache Hadoop Maven Plugins …………………… SUCCESS [ 14.448 s]

    [INFO] Apache Hadoop MiniKDC ………………………… SUCCESS [ 25.414 s]

    [INFO] Apache Hadoop Auth …………………………… SUCCESS [ 15.027 s]

    [INFO] Apache Hadoop Auth Examples …………………… SUCCESS [ 7.052 s]

    [INFO] Apache Hadoop Common …………………………. SUCCESS [02:01 min]

    [INFO] Apache Hadoop NFS ……………………………. SUCCESS [ 8.556 s]

    [INFO] Apache Hadoop KMS ……………………………. SUCCESS [02:31 min]

    [INFO] Apache Hadoop Common Project ………………….. SUCCESS [ 0.029 s]

    [INFO] Apache Hadoop HDFS …………………………… SUCCESS [02:46 min]

    [INFO] Apache Hadoop HttpFS …………………………. SUCCESS [02:42 min]

    [INFO] Apache Hadoop HDFS BookKeeper Journal ………….. SUCCESS [ 14.788 s]

    [INFO] Apache Hadoop HDFS-NFS ……………………….. SUCCESS [ 3.519 s]

    [INFO] Apache Hadoop HDFS Project ……………………. SUCCESS [ 0.029 s]

    [INFO] hadoop-yarn …………………………………. SUCCESS [ 0.037 s]

    [INFO] hadoop-yarn-api ……………………………… SUCCESS [01:12 min]

    [INFO] hadoop-yarn-common …………………………… SUCCESS [ 25.486 s]

    [INFO] hadoop-yarn-server …………………………… SUCCESS [ 0.044 s]

    [INFO] hadoop-yarn-server-common …………………….. SUCCESS [ 11.236 s]

    [INFO] hadoop-yarn-server-nodemanager ………………… SUCCESS [ 19.452 s]

    [INFO] hadoop-yarn-server-web-proxy ………………….. SUCCESS [ 2.385 s]

    [INFO] hadoop-yarn-server-applicationhistoryservice ……. SUCCESS [ 5.028 s]

    [INFO] hadoop-yarn-server-resourcemanager …………….. SUCCESS [ 15.531 s]

    [INFO] hadoop-yarn-server-tests ……………………… SUCCESS [ 4.019 s]

    [INFO] hadoop-yarn-client …………………………… SUCCESS [ 5.003 s]

    [INFO] hadoop-yarn-applications ……………………… SUCCESS [ 0.044 s]

    [INFO] hadoop-yarn-applications-distributedshell ………. SUCCESS [ 2.060 s]

    [INFO] hadoop-yarn-applications-unmanaged-am-launcher ….. SUCCESS [ 1.525 s]

    [INFO] hadoop-yarn-site …………………………….. SUCCESS [ 0.047 s]

    [INFO] hadoop-yarn-registry …………………………. SUCCESS [ 4.122 s]

    [INFO] hadoop-yarn-project ………………………….. SUCCESS [ 5.211 s]

    [INFO] hadoop-mapreduce-client ………………………. SUCCESS [ 0.051 s]

    [INFO] hadoop-mapreduce-client-core ………………….. SUCCESS [ 16.098 s]

    [INFO] hadoop-mapreduce-client-common ………………… SUCCESS [ 12.579 s]

    [INFO] hadoop-mapreduce-client-shuffle ……………….. SUCCESS [ 3.310 s]

    [INFO] hadoop-mapreduce-client-app …………………… SUCCESS [ 6.972 s]

    [INFO] hadoop-mapreduce-client-hs ……………………. SUCCESS [ 5.790 s]

    [INFO] hadoop-mapreduce-client-jobclient ……………… SUCCESS [ 7.923 s]

    [INFO] hadoop-mapreduce-client-hs-plugins …………….. SUCCESS [ 1.437 s]

    [INFO] Apache Hadoop MapReduce Examples ………………. SUCCESS [ 4.272 s]

    [INFO] hadoop-mapreduce …………………………….. SUCCESS [ 4.270 s]

    [INFO] Apache Hadoop MapReduce Streaming ……………… SUCCESS [ 5.100 s]

    [INFO] Apache Hadoop Distributed Copy ………………… SUCCESS [ 12.644 s]

    [INFO] Apache Hadoop Archives ……………………….. SUCCESS [ 1.797 s]

    [INFO] Apache Hadoop Rumen ………………………….. SUCCESS [ 4.551 s]

    [INFO] Apache Hadoop Gridmix ………………………… SUCCESS [ 3.380 s]

    [INFO] Apache Hadoop Data Join ………………………. SUCCESS [ 2.232 s]

    [INFO] Apache Hadoop Ant Tasks ………………………. SUCCESS [ 2.054 s]

    [INFO] Apache Hadoop Extras …………………………. SUCCESS [ 2.312 s]

    [INFO] Apache Hadoop Pipes ………………………….. SUCCESS [ 8.269 s]

    [INFO] Apache Hadoop OpenStack support ……………….. SUCCESS [ 4.217 s]

    [INFO] Apache Hadoop Amazon Web Services support ………. SUCCESS [ 29.522 s]

    [INFO] Apache Hadoop Client …………………………. SUCCESS [ 7.187 s]

    [INFO] Apache Hadoop Mini-Cluster ……………………. SUCCESS [ 0.106 s]

    [INFO] Apache Hadoop Scheduler Load Simulator …………. SUCCESS [ 3.483 s]

    [INFO] Apache Hadoop Tools Dist ……………………… SUCCESS [ 8.224 s]

    [INFO] Apache Hadoop Tools ………………………….. SUCCESS [ 0.025 s]

    [INFO] Apache Hadoop Distribution ……………………. SUCCESS [ 49.890 s]

    [INFO] ————————————————————————

    [INFO] BUILD SUCCESS

    [INFO] ————————————————————————

    [INFO] Total time: 18:13 min

    [INFO] Finished at: 2017-06-23T03:56:38+08:00

    [INFO] Final Memory: 157M/350M

    [INFO] ————————————————————————

    截图如下:

    redhat7源码编译hadoop2.6.0

    如果不通过源码编译安装,在hadoop,spark,flink等启动时经常会提示一个警告:

    2017-06-23 21:57:49,105 WARN org.apache.hadoop.util.NativeCodeLoader – Unable to load native-hadoop library for your platform… using builtin-java classes where applicable

    喜欢 (0)
    [1353713598@qq.com]
    分享 (0)
    发表我的评论
    取消评论
    表情 贴图 加粗 删除线 居中 斜体 签到

    Hi,您需要填写昵称和邮箱!

    • 昵称 (必填)
    • 邮箱 (必填)
    • 网址