sqoop導出hive數據到mysql報錯:Causedby:java.lang.RuntimeException:Can'tparseinputdata的解決辦法

Sqoop Export數據到本地資料庫時出現錯誤,命令如下:

sqoop export \
--connect 'jdbc:mysql://202.193.60.117/dataweb?useUnicode=true&characterEncoding=utf-8' \
--username root \
--password-file /user/hadoop/.password \
--table user_info_copy \
--export-dir /user/hadoop/user_info \
--input-fields-terminated-by "@"

錯誤日志如下:

Error: java.io.IOException: Can't export data, please check failed map task logs
    at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
    at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
    at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.RuntimeException: Can't parse input data: '2,hello,456,0'
    at user_info_copy.__loadFromFields(user_info_copy.java:335)
    at user_info_copy.parse(user_info_copy.java:268)
    at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83)
    ... 10 more
Caused by: java.lang.NumberFormatException: For input string: "2,hello,456,0"
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
    at java.lang.Integer.parseInt(Integer.java:580)
    at java.lang.Integer.valueOf(Integer.java:766)
    at user_info_copy.__loadFromFields(user_info_copy.java:317)
    ... 12 more

該錯誤的問題在於分隔符,因為您沒有在配置單元表中指定分隔符,默認的配置單元分隔符是:

‘\ 001’字段終止

‘\ 002’的收藏品終止

‘\ 003’終止的地圖密鑰

LINES終止於’\ n’

所以要解決問題,就需要更改配置單元創建或sqoop作業中的分隔符來解決問題。在本問題中,我的表示在本地通過界面化管理工具創建的,分隔符換成“,”就能解決問題瞭,若是其他分隔符則對應改變就行;或者,使用以下語句重新創建表格並指定分隔符來解決。

ROW FORMAT DELIMITED FIELDS TERMINATED BY '分隔符'

發佈留言

發佈留言必須填寫的電子郵件地址不會公開。 必填欄位標示為 *