sqoop报错is running 8724480B beyond the 'PHYSICAL' memory limit.

is running 8724480B beyond the 'PHYSICAL' memory limit.

解决方法：

sqoop同步数据量稍微有点大时就会包内存不足，可以把时间段拆开几段时间来同步。

[2022-04-28 16:28:09.331]Container [pid=32899,containerID=container_e67_1641446154228_152726_01_000004] is running 8724480B beyond the 'PHYSICAL' memory limit. Current usage: 1.0 GB of 1 GB physical memory used; 2.5 GB of 2.1 GB virtual memory used. Killing container.

假如我要把2017-01-01到2018-01-01时间段按6个月拆开执行脚本

#!/bin/bash

startDate=20170101

endDate=20180101

startSec=`date -d "$startDate" "+%s"`

endSec=`date -d "$endDate" "+%s"`

for((i=$startSec;i<=$endSec;))

do

  echo $i

   firstday=`date -d "@$i" "+%Y-%m-%d"`

   echo ${firstday}

   secondday=`date -d "${firstday} 6 month" "+%Y-%m-%d"`

   secondSec=`date -d "${firstday} 6 month" "+%s"`

   echo ${secondday}

   i=${secondSec}

 echo "此处放sqoop同步脚本"

 done

顶

0