Sqoop Export數據到本地資料庫時出現錯誤,命令如下:
sqoop export \ --connect 'jdbc:mysql://202.193.60.117/dataweb?useUnicode=true&characterEncoding=utf-8' \ --username root \ --password-file /user/hadoop/.password \ --table user_info_copy \ --export-dir /user/hadoop/user_info \ --input-fields-terminated-by "@"
錯誤日志如下:
Error: java.io.IOException: Can't export data, please check failed map task logs at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112) at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: java.lang.RuntimeException: Can't parse input data: '2,hello,456,0' at user_info_copy.__loadFromFields(user_info_copy.java:335) at user_info_copy.parse(user_info_copy.java:268) at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83) ... 10 more Caused by: java.lang.NumberFormatException: For input string: "2,hello,456,0" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:580) at java.lang.Integer.valueOf(Integer.java:766) at user_info_copy.__loadFromFields(user_info_copy.java:317) ... 12 more
該錯誤的問題在於分隔符,因為您沒有在配置單元表中指定分隔符,默認的配置單元分隔符是:
‘\ 001’字段終止
‘\ 002’的收藏品終止
‘\ 003’終止的地圖密鑰
LINES終止於’\ n’
所以要解決問題,就需要更改配置單元創建或sqoop作業中的分隔符來解決問題。在本問題中,我的表示在本地通過界面化管理工具創建的,分隔符換成“,”就能解決問題瞭,若是其他分隔符則對應改變就行;或者,使用以下語句重新創建表格並指定分隔符來解決。
ROW FORMAT DELIMITED FIELDS TERMINATED BY '分隔符'