[
https://issues.apache.org/jira/browse/ASTERIXDB-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yipzifai updated ASTERIXDB-1478:
--------------------------------
Description:
I load data file using localfs but AsterixDB keeps running for a long long
time.(ps:CPU utilization ~= 100%) I have been waiting over 8 hours
yesterday.Then i found that the storage folder in workingdir didn't add any
files.
Exectly, I try to use feeding or AQL "Insert", but it still goes wrong.
AQL:
use dataverse test;
drop dataset relationship_B if exists;
drop type relationshipB if exists;
create type relationshipB as closed{
pkId:int64,
who_id:int64,
who_name:string,
id:int64,
name:string,
code:string,
shortname:string,
brief:string,
validateDate:date?,
gfType:string,
shijizengjianchi:string,
chiyoushuliang:double,
chigubianhua:string,
zhanbi:double,
konggugongsiID:int64?,
konggugongsiName:string?,
rType:string,
source:string,
method:string
};
create dataset relationship_B(relationshipB) primary key pkId;
use dataverse test;
load dataset relationship_B
using localfs
(("path"="127.0.0.1:///home/to/adbSrc/ddl/relationshipB.adm"),("format"="adm"));
About data files:
one file is big object insertion(each record has over 30,000 characters); each
record of another file has 100~200 characters.
{code}
local.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<cluster xmlns="cluster">
<java_home>/usr/local/jdk1.8.0_91/jre</java_home>
<log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/logs</log_dir>
<txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/txnLogs</txn_log_dir>
<store>storage</store>
<working_dir>
<dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir</dir>
<NFS>true</NFS>
</working_dir>
<iodevices>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc/p1,/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc/p2</iodevices>
<master_node>
<id>master</id>
<client_ip>127.0.0.1</client_ip>
<cluster_ip>127.0.0.1</cluster_ip>
<client_port>1098</client_port>
<cluster_port>1099</cluster_port>
<http_port>8888</http_port>
<debug_port>8900</debug_port>
</master_node>
<node>
<id>nc1</id>
<cluster_ip>127.0.0.1</cluster_ip>
<txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc1/txnLogs</txn_log_dir>
<debug_port>8901</debug_port>
</node>
<node>
<id>nc2</id>
<cluster_ip>127.0.0.1</cluster_ip>
<txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc2/txnLogs</txn_log_dir>
<debug_port>8902</debug_port>
</node>
<heartbeat_period>10000</heartbeat_period>
<max_heartbeat_lapse_periods>5</max_heartbeat_lapse_periods>
<profile_dump_period>0</profile_dump_period>
<default_max_job_attempts>5</default_max_job_attempts>
<job_history_size>10</job_history_size>
<result_time_to_live>86400000</result_time_to_live>
<result_sweep_threshold>60000</result_sweep_threshold>
</cluster>
{code}
was:
I load data file using localfs but AsterixDB keeps running for a long long
time.(ps:CPU utilization ~= 100%) I have been waiting over 8 hours
yesterday.Then i found that the storage folder in workingdir didn't add any
files.
Exectly, I try to use feeding or AQL "Insert", but it still goes wrong.
About data files:one file is big object insertion(each record has over 30,000
characters); records' size of another file has 100~200 characters.
{code}
local.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<cluster xmlns="cluster">
<java_home>/usr/local/jdk1.8.0_91/jre</java_home>
<log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/logs</log_dir>
<txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/txnLogs</txn_log_dir>
<store>storage</store>
<working_dir>
<dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir</dir>
<NFS>true</NFS>
</working_dir>
<iodevices>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc/p1,/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc/p2</iodevices>
<master_node>
<id>master</id>
<client_ip>127.0.0.1</client_ip>
<cluster_ip>127.0.0.1</cluster_ip>
<client_port>1098</client_port>
<cluster_port>1099</cluster_port>
<http_port>8888</http_port>
<debug_port>8900</debug_port>
</master_node>
<node>
<id>nc1</id>
<cluster_ip>127.0.0.1</cluster_ip>
<txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc1/txnLogs</txn_log_dir>
<debug_port>8901</debug_port>
</node>
<node>
<id>nc2</id>
<cluster_ip>127.0.0.1</cluster_ip>
<txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc2/txnLogs</txn_log_dir>
<debug_port>8902</debug_port>
</node>
<heartbeat_period>10000</heartbeat_period>
<max_heartbeat_lapse_periods>5</max_heartbeat_lapse_periods>
<profile_dump_period>0</profile_dump_period>
<default_max_job_attempts>5</default_max_job_attempts>
<job_history_size>10</job_history_size>
<result_time_to_live>86400000</result_time_to_live>
<result_sweep_threshold>60000</result_sweep_threshold>
</cluster>
{code}
> using localfs goes wrong
> ------------------------
>
> Key: ASTERIXDB-1478
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1478
> Project: Apache AsterixDB
> Issue Type: Bug
> Components: Error Reporting
> Environment: AsterixDB 0.8.9(pulled from github today)
> Ubuntu 14.0
> Java -version 1.8.0_79
> Computer has 1 ssd and 1 hhd
> Reporter: yipzifai
> Labels: features
>
> I load data file using localfs but AsterixDB keeps running for a long long
> time.(ps:CPU utilization ~= 100%) I have been waiting over 8 hours
> yesterday.Then i found that the storage folder in workingdir didn't add any
> files.
> Exectly, I try to use feeding or AQL "Insert", but it still goes wrong.
> AQL:
> use dataverse test;
> drop dataset relationship_B if exists;
> drop type relationshipB if exists;
> create type relationshipB as closed{
> pkId:int64,
> who_id:int64,
> who_name:string,
> id:int64,
> name:string,
> code:string,
> shortname:string,
> brief:string,
> validateDate:date?,
> gfType:string,
> shijizengjianchi:string,
> chiyoushuliang:double,
> chigubianhua:string,
> zhanbi:double,
> konggugongsiID:int64?,
> konggugongsiName:string?,
> rType:string,
> source:string,
> method:string
> };
> create dataset relationship_B(relationshipB) primary key pkId;
> use dataverse test;
> load dataset relationship_B
> using localfs
> (("path"="127.0.0.1:///home/to/adbSrc/ddl/relationshipB.adm"),("format"="adm"));
> About data files:
> one file is big object insertion(each record has over 30,000 characters);
> each record of another file has 100~200 characters.
> {code}
> local.xml
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <cluster xmlns="cluster">
> <java_home>/usr/local/jdk1.8.0_91/jre</java_home>
>
> <log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/logs</log_dir>
>
> <txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/txnLogs</txn_log_dir>
> <store>storage</store>
> <working_dir>
> <dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir</dir>
> <NFS>true</NFS>
> </working_dir>
>
> <iodevices>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc/p1,/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc/p2</iodevices>
> <master_node>
> <id>master</id>
> <client_ip>127.0.0.1</client_ip>
> <cluster_ip>127.0.0.1</cluster_ip>
> <client_port>1098</client_port>
> <cluster_port>1099</cluster_port>
> <http_port>8888</http_port>
> <debug_port>8900</debug_port>
> </master_node>
> <node>
> <id>nc1</id>
> <cluster_ip>127.0.0.1</cluster_ip>
>
> <txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc1/txnLogs</txn_log_dir>
>
> <debug_port>8901</debug_port>
> </node>
> <node>
> <id>nc2</id>
> <cluster_ip>127.0.0.1</cluster_ip>
>
> <txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc2/txnLogs</txn_log_dir>
>
> <debug_port>8902</debug_port>
> </node>
> <heartbeat_period>10000</heartbeat_period>
> <max_heartbeat_lapse_periods>5</max_heartbeat_lapse_periods>
> <profile_dump_period>0</profile_dump_period>
> <default_max_job_attempts>5</default_max_job_attempts>
> <job_history_size>10</job_history_size>
> <result_time_to_live>86400000</result_time_to_live>
> <result_sweep_threshold>60000</result_sweep_threshold>
> </cluster>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
