sqoop数据量大怎么同步
解决方法：
sqoop同步数据量稍微有点大时就会包内存不足，可以把时间段拆开几段时间来同步。
错误信息：
[2022-04-28 16:28:09.331]Container [pid=32899,containerID=container_e67_1641446154228_152726_01_000004] is running 8724480B beyond the 'PHYSICAL' memory limit. Current usage: 1.0 GB of 1 GB physical memory used; 2.5 GB of 2.1 GB virtual memory used. Killing container.
假如我要把2017-01-01到2018-01-01时间段按6个月拆开执行脚本
#!/bin/bash
startDate=20170101
endDate=20180101
startSec=`date -d "$startDate" "+%s"`
endSec=`date -d "$endDate" "+%s"`
for((i=$startSec;i<=$endSec;))
do
echo $i
firstday=`date -d "@$i" "+%Y-%m-%d"`
echo ${firstday}
secondday=`date -d "${firstday} 6 month" "+%Y-%m-%d"`
secondSec=`date -d "${firstday} 6 month" "+%s"`
echo ${secondday}
i=${secondSec}
echo "此处放sqoop同步脚本"
done