错误信息:

2022-12-11 22:02:39,468 ERROR [org.apache.hadoop.util.Shell] - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable D:\Development_tools\Hadoop\hadoop-3.3.4\bin\winutils.exe in the Hadoop binaries.
	at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:356)
	at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:371)
	at org.apache.hadoop.util.Shell.<clinit>(Shell.java:364)
	at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:80)
	at org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
	at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:272)
	at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:260)
	at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:790)
	at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:760)
	at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:633)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.<init>(JobContextImpl.java:72)
	at org.apache.hadoop.mapreduce.Job.<init>(Job.java:142)
	at org.apache.hadoop.mapreduce.Job.getInstance(Job.java:185)
	at com.hadoop.workcount.WordCountDriver.main(WordCountDriver.java:19)

解决办法:

出现此问题是因为:在hadoop的bin目录下找不到可执行文件 bin\winutils.exe。因此将winutils.exe文件放入bin目录下即可解决:winutils.exe资源包

GitHub 加速计划 / wi / winutils
5
1
下载
Windows binaries for Hadoop versions (built from the git commit ID used for the ASF relase)
最近提交(Master分支:3 个月前 )
e8089ecf - 3 年前
d4f71517 point people at cdarlint/winutils for binaries and call out the fact that we could remove the need for this entirely just to run spark on windows 6 年前
Logo

AtomGit 是由开放原子开源基金会联合 CSDN 等生态伙伴共同推出的新一代开源与人工智能协作平台。平台坚持“开放、中立、公益”的理念,把代码托管、模型共享、数据集托管、智能体开发体验和算力服务整合在一起,为开发者提供从开发、训练到部署的一站式体验。

更多推荐