码上敲享录 > 大数据常见问题解答 > sqoop报错is running 8724480B beyond the 'PHYSICAL' memory limit.

sqoop报错is running 8724480B beyond the 'PHYSICAL' memory limit.

上一章章节目录下一章 2022-04-29已有356人阅读 评论(0)

is running 8724480B beyond the 'PHYSICAL' memory limit.


解决方法:

sqoop同步数据量稍微有点大时就会包内存不足,可以把时间段拆开几段时间来同步。

[2022-04-28 16:28:09.331]Container [pid=32899,containerID=container_e67_1641446154228_152726_01_000004] is running 8724480B beyond the 'PHYSICAL' memory limit. Current usage: 1.0 GB of 1 GB physical memory used; 2.5 GB of 2.1 GB virtual memory used. Killing container.

假如我要把2017-01-01到2018-01-01时间段按6个月拆开执行脚本

#!/bin/bash

startDate=20170101

endDate=20180101

startSec=`date -d "$startDate" "+%s"`

endSec=`date -d "$endDate" "+%s"`

for((i=$startSec;i<=$endSec;))

do

  echo $i

   firstday=`date -d "@$i" "+%Y-%m-%d"`

   echo ${firstday}

   secondday=`date -d "${firstday} 6 month" "+%Y-%m-%d"`

   secondSec=`date -d "${firstday} 6 month" "+%s"`

   echo ${secondday}

   i=${secondSec}

 echo "此处放sqoop同步脚本"

 done


向大家推荐《Activiti工作流实战教程》:https://xiaozhuanlan.com/activiti
0

有建议,请留言!

  • *您的姓名:

  • *所在城市:

  • *您的联系电话:

    *您的QQ:

  • 咨询问题:

  • 提 交