Joinutility seperatorLogin utility separator Infobright.com
   
1 of 3
1
Problems using LOAD DATA INFILE—can anyone help me?
Posted: 27 February 2009 03:47 PM   Ignore ]  
Newbie
Rank
Total Posts:  19
Joined  2009-01-24

Hi all,

just pulled ICE 3.1 for Windows and running it on XP x64 but now I’m trying to load data—I have this issue here:

—when I try this:
LOAD DATA LOCAL INFILE ‘C:\ WO.TXT ‘ INTO TABLE WO FIELDS TERMINATED BY ‘,’ LINES TERMINATED BY ‘\n’ IGNORE 1 LINES;

I get this:
<eb1>Brighthouse: Session terminated unexpectedly, uncommitted transactions rolled back.
State: S1000; Native: 1402; Origin: [MySQL][ODBC 5.1 Driver] [mysqld-5.1.14-log]</eb1 >

Looked about everywhere on Google and ICE forums/wiki but cant see what the problem is—anything objvious?

Thanks!

Profile
 
Posted: 27 February 2009 04:12 PM   Ignore ]   [ # 1 ]  
Sr. Member
Avatar
RankRankRankRank
Total Posts:  505
Joined  2008-08-18
XSPRADA_MAN - 27 February 2009 03:47 PM

Hi all,

just pulled ICE 3.1 for Windows and running it on XP x64 but now I’m trying to load data—I have this issue here:

—when I try this:
LOAD DATA LOCAL INFILE ‘C:\ WO.TXT ‘ INTO TABLE WO FIELDS TERMINATED BY ‘,’ LINES TERMINATED BY ‘\n’ IGNORE 1 LINES;

I get this:
<eb1>Brighthouse: Session terminated unexpectedly, uncommitted transactions rolled back.
State: S1000; Native: 1402; Origin: [MySQL][ODBC 5.1 Driver] [mysqld-5.1.14-log]</eb1 >

Looked about everywhere on Google and ICE forums/wiki but cant see what the problem is—anything objvious?

Thanks!

Hi,

I see a couple of things:

1) We don’t support LOAD DATA LOCAL INFILE today. Use LOAD DATA INFILE instead. The load file needs to be present on the same server where the database is running.

We are looking to support LOCAL INFILE in the future, but it’s not there yet.

2) Use forward slashes in the file path: ‘C:/WO.TXT’

In the load command, backslash is interpreted as an escape character.

3) IGNORE 1 LINES isn’t supported today. Delete the header line from the file and try the load again.

Let us know if this helps…

Geoffrey

Signature 
Profile
 
Posted: 27 February 2009 04:15 PM   Ignore ]   [ # 2 ]  
Newbie
Rank
Total Posts:  19
Joined  2009-01-24

Yes I had just tried changing the path name succesfully and so it works as such now:

LOAD DATA INFILE ‘/../../WO.TXT’ INTO TABLE WO FIELDS TERMINATED BY ‘,’ LINES TERMINATED BY ‘\n’ ;


using a relative path.  So I load the data and do a select count(*) and it has all the rows however on a select * I get this informative message:

<eb1>An unknown error has occurred.</eb1>

Anyway to examine what’s in that table?

BTW I’m using QTODBC 7.0 with the MySQL ODBC driver 5.1

Thanks!

Profile
 
Posted: 27 February 2009 04:58 PM   Ignore ]   [ # 3 ]  
Sr. Member
Avatar
RankRankRankRank
Total Posts:  505
Joined  2008-08-18
XSPRADA_MAN - 27 February 2009 04:15 PM

Yes I had just tried changing the path name succesfully and so it works as such now:

LOAD DATA INFILE ‘/../../WO.TXT’ INTO TABLE WO FIELDS TERMINATED BY ‘,’ LINES TERMINATED BY ‘\n’ ;


using a relative path.  So I load the data and do a select count(*) and it has all the rows however on a select * I get this informative message:

<eb1>An unknown error has occurred.</eb1>

Anyway to examine what’s in that table?

BTW I’m using QTODBC 7.0 with the MySQL ODBC driver 5.1

Thanks!

Maybe the data got corrupted by some previous errors. Try dropping and recreating the table and do the load again.

Also look in the error log (brighthouse.log, bh.err) and see if there’s any info there.

Geoffrey

Signature 
Profile
 
Posted: 27 February 2009 05:26 PM   Ignore ]   [ # 4 ]  
Newbie
Rank
Total Posts:  19
Joined  2009-01-24

Yup, done all that—same results.

Profile
 
Posted: 27 February 2009 06:01 PM   Ignore ]   [ # 5 ]  
Jr. Member
Avatar
RankRank
Total Posts:  96
Joined  2008-08-18

Please provide us with some sample data, your create table statement, and your load statement.

Signature 

Brian Beharry, QA
Infobright

Profile
 
Posted: 27 February 2009 06:02 PM   Ignore ]   [ # 6 ]  
Administrator
RankRankRank
Total Posts:  139
Joined  2008-08-18

Hi XSPRADA_MAN,

Can send us the error file bh.err, general_query.log from data directory. Also if you can send your data we can try loading it.

thanks,
kethees

Profile
 
Posted: 27 February 2009 06:04 PM   Ignore ]   [ # 7 ]  
Newbie
Rank
Total Posts:  19
Joined  2009-01-24

I cant give you any data as it’s live data and proprietary from a client.  (Besides it’s way too large)—it’s contained in a CSV flat file.

DDL is here:

CREATE TABLE wo (
  REPORT_DATE_INT int ,
  HO_WORK_ORDER_HISTORY_ID int ,
  HO_PHONE_MASTER_ROW_ID int ,
  HO_SUBSCRIBER_ROW_ID int ,
  SUB_CTYPE varchar(5) ,
  WKH_RCD char(3) ,
  WKH_DISC_REASON varchar(20) ,
  MARKET_CODE varchar(4) ,
  SALE_TYPE char(3)  ,
  STORE_CHANNEL_ID int ,
  WKH_SALESMAN varchar(10) ,
  WKH_CSR_INIT varchar(20) ,
  FGG smallint ,
  RETURNS smallint ,
  REACTS smallint ,
  TRANSFERS smallint ,
  GG smallint ,
  GROSS_ADDS smallint ,
  ADJ_GROSS_ADDS smallint ,
  GROSS_DEACTS smallint ,
  DEACTS smallint ,
  ADJ_DEACTS smallint ,
  NET_ADDS smallint ,
  REPORT_DATE timestamp ,
  IMPORT_FILE_HISTORY_ID int ,
  DATE_CREATED timestamp ,
  WKH_D_RATE_PLAN_ID smallint ,
  WKH_A_RATE_PLAN_ID smallint ,
  PH_RATE_PLAN_ID smallint ,
  WBK_ADDS smallint ,
  WBK_REACTS smallint ,
  WBK_DEACT smallint ,
  CRTALK_DEACT smallint ,
  WKH_CAMPAIGN_ID smallint ,
  SA smallint ,
  CPE smallint ,
  ESN_ACT varchar(30) ,
  ESN_DEACT varchar(30) ,
  PAR smallint ,
  PAR_ADDR smallint ,
  PAR_SSN smallint ,
  PAR_DRV smallint ,
  PAR_BAL smallint ,
  PAR_DOB smallint ,
  TRANSFER_DEACT smallint ,
  HO_CC_BILL char(1) ,
  SUB_ISSUE_BILL char(1) ,
  Cricket_Location_ID_Key int ,
  Store_Sale_Type char(3) ,
  MIGRATIONS smallint ,
  TRANSFERS_PREPAID smallint ,
  TRANSFERS_POSTPAID smallint ,
  TRANSFERS_OTHER smallint ,
  A_FEATURE_SET_ID int ,
  D_FEATURE_SET_ID int ,
  PHONE_FEATURE_SET_ID int ,
  IS_CRICKET_HANDSET smallint );

I am loading with:

LOAD DATA INFILE ‘/../../WO.TXT’ INTO TABLE WO FIELDS TERMINATED BY ‘,’ LINES TERMINATED BY ‘\n’ ;

Profile
 
Posted: 27 February 2009 06:42 PM   Ignore ]   [ # 8 ]  
Administrator
RankRankRank
Total Posts:  139
Joined  2008-08-18

Thanks for the DDL. We will try to reproduce it here. In the mean time, can you try loading a smaller set of data like 10 lines and selecting from that.

We have tested the 32 bit version on a 32 bit machine but not on 64. Is there is anything in the error file or general_query log file?

Profile
 
Posted: 27 February 2009 06:43 PM   Ignore ]   [ # 9 ]  
Newbie
Rank
Total Posts:  19
Joined  2009-01-24

Yeah that’s what I first did, with about 57 rows. Same problem.

It runs 32 bits on the WOW64 emulator anyway right? 

Nothing of interest in the logs—i checked em all.

Thanks for the help.
Jerome.

Profile
 
Posted: 28 February 2009 05:45 PM   Ignore ]   [ # 10 ]  
Newbie
Rank
Total Posts:  19
Joined  2009-01-24

Ok so I think it’s a CSV format issue.  I simplified it down to this:

CREATE TABLE jerome (
  REPORT_DATE_INT int ,
  HO_WORK_ORDER_HISTORY_ID int ,
  HO_PHONE_MASTER_ROW_ID int ,
  HO_SUBSCRIBER_ROW_ID int ,
  SUB_CTYPE varchar(5)
);


It’s when it gets to the last col there’s a problem.

The CSV is this:
20080401,157290848,3763,970079,“01”

with a CR-LF at the end.

Also tried
20080401,157290848,3763,970079,01

to no avail.

But it’s when I add that varchar(5) at the end that it blows up on SELECT * FROM…

Is there documented direction somewhere on how to best format CSV for presentation to the ICE/mySQL engine?

Thanks.

Profile
 
Posted: 02 March 2009 11:46 AM   Ignore ]   [ # 11 ]  
Sr. Member
Avatar
RankRankRankRank
Total Posts:  505
Joined  2008-08-18
XSPRADA_MAN - 27 February 2009 06:04 PM

I am loading with:

LOAD DATA INFILE ‘/../../WO.TXT’ INTO TABLE WO FIELDS TERMINATED BY ‘,’ LINES TERMINATED BY ‘\n’ ;

Try putting

OPTIONALLY ENCLOSED BY '"' 

in the load command.

Signature 
Profile
 
Posted: 02 March 2009 12:05 PM   Ignore ]   [ # 12 ]  
Newbie
Rank
Total Posts:  19
Joined  2009-01-24

Tried:
LOAD DATA INFILE ‘/../../jerome.txt’ INTO TABLE jerome FIELDS TERMINATED BY ‘,’ LINES TERMINATED BY ‘\n’ OPTIONALLY ENCLOSED BY ‘“’

Got:
<eb1>You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘OPTIONALLY ENCLOSED BY ‘“’’ at line 1
State: 37000; Native: 1064; Origin: [MySQL][ODBC 5.1 Driver][mysqld-5.1.14-log]</eb1>

Profile
 
Posted: 02 March 2009 12:09 PM   Ignore ]   [ # 13 ]  
Sr. Member
Avatar
RankRankRankRank
Total Posts:  505
Joined  2008-08-18

OPTIONALLY ENCLOSED BY applies to the FIELDS section.. try this

LOAD DATA INFILE '/../../jerome.txt' INTO TABLE jerome FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"' LINES TERMINATED BY '\n' 

Geoffrey

Signature 
Profile
 
Posted: 02 March 2009 12:12 PM   Ignore ]   [ # 14 ]  
Newbie
Rank
Total Posts:  19
Joined  2009-01-24

Thanks but same result

changed data to:

20080401,157290848,3763,970079,“01”

Schema:
CREATE TABLE jerome (
  REPORT_DATE_INT int ,
  HO_WORK_ORDER_HISTORY_ID int ,
  HO_PHONE_MASTER_ROW_ID int ,
  HO_SUBSCRIBER_ROW_ID int ,
  SUB_CTYPE varchar(5)
);

select * from jerome yields:

<eb1>An unknown error has occurred.</eb1>

Profile
 
Posted: 02 March 2009 12:42 PM   Ignore ]   [ # 15 ]  
Jr. Member
Avatar
RankRank
Total Posts:  96
Joined  2008-08-18

Jerome,

When I use this data generator, and I configure the data generator like this:

<xml>
    <
row type="delimited" seperator=";">
        <
field type="regex" pattern="[+\-]{1}[0-9]{8}" length="9" />
        <
field type="regex" pattern="[+\-]{1}[0-9]{8}" length="9" />
        <
field type="regex" pattern="[+\-]{1}[0-9]{8}" length="9" />
        <
field type="regex" pattern="[+\-]{1}[0-9]{8}" length="9" />
        <
field type="regex" pattern="[a-zA-Z0-9]{5}" length="5" />
    </
row>
</
xml

Then, I can load and select with these commands:

warnings;

drop database if exists jerome_test;
create database jerome_test;
use 
jerome_test;


set autocommit=1;


drop table if exists jerome;
CREATE TABLE jerome (
  
REPORT_DATE_INT int ,
  
HO_WORK_ORDER_HISTORY_ID int ,
  
HO_PHONE_MASTER_ROW_ID int ,
  
HO_SUBSCRIBER_ROW_ID int ,
  
SUB_CTYPE varchar(5)
);


set @bh_dataformat 'txt_variable';
--  
The next command is all on one line
load data infile 
'C:/downloads/datagenerator0.26/datagenerator/randomdata.txt' into table jerome       fields terminated by ';' enclosed by 'null' lines terminated by '\n';

select count(*) from jerome;
select from jerome

Here’s a sample of the data I generated (can you load and select from this with my statements above?):

+12717906;-84816883;+17796263;-83720239;gvqJL
+65857363;+19935432;+32836681;+21714780;zAMqL
-89088488;-46366250;+26632425;+20053433;uTh7l
+32159212;-01959562;+60433380;-57249608;LlSa5
-50609942;-79496980;-07003982;-43424357;28gOm
-75228283;-95647777;-34363516;-65064622;FqE0B
-86679095;-73179209;-35948226;+92371310;Npks9
-95987068;+19862786;-46408020;+35140167;XJgW0
+05535196;+09964138;-49303840;-69488450;T4s4j
-86752063;+36505309;-68984498;-48814321;4jWjU
+66822788;-24019431;-21156385;+52570375;IeTIK
+28406252;+33804510;+39221891;-09240801;jKarL
+90653760;-91733482;+61556878;-22901829;LDCcI
-48774030;-26057436;+28826012;-24878173;eB7L2
-98363212;+30109121;-74244153;-31569027;UvcT1
+47772084;-88693302;-08792545;+16444895;M64I6
+26838746;-34068082;+96781822;-53741703;iop6X
+73105973;-75876310;+22549216;+47255886;pznVc
+67313329;-86382811;-63630309;-97861672;rHvNJ
+46801655;-19358979;+47411782;+99896794;DJdgp
+29118959;+82164490;-97516835;+14271247;96WAh
-04670012;+46852207;+17851242;-32720445;KFYS6
+88609753;-76430476;+48604755;+57084927;P5tzv
-90830014;+00430916;+01341909;-61234483;fkNiA
+75104731;+17984200;+90845941;+72231609;IIaA5
+43988259;+70923136;-42686276;+38739253;CDViq
-50987363;+89761604;-62448625;-43040176;zZVe6
+91796802;+49384711;-30767443;+06708160;bd2NS
+50061699;+53806920;-72265548;+57394426;8sGy7
-37646218;-34886738;-28893883;+55823909;nBrLW
+46926672;+28312450;-29336351;+24168618;hfWcn
+34521365;-57784466;+88509265;-69192744;eNDel
-89866764;+14906015;-08117864;-51314358;eZRIW
-46502849;-94632905;-10777207;+93617650;aoISJ
-73812288;-46160934;-44515244;+70203434;K1fsw
-81720255;-48007122;+12463327;+20487384;CpMXr
-34456089;-35448450;-19869341;+87794401;OVcx9
-98419204;-16881028;+51702856;+08076697;IuvdP
-13839213;-08345100;+63816186;-97437939;eHAsK
-54823771;-47284564;+61732420;+20001900;54Fzb
+40627678;-66038769;+84813529;-34536102;k22NS
+55212677;+25948982;+61392527;-33101690;NtKHd
+30844183;-93610988;+18935316;+09977715;xXmfq
+89775622;+37663352;+20734081;-55554624;Wo7IJ
-58548569;-72285890;+54752882;+26907795;xfdkr
+38002928;+45018566;-31181173;+78005614;wY0Rv
-34765690;-32642496;-74747483;+18363199;ao1WA
-19166120;-39590236;+92013183;+39068914;iOYLg
+36009480;+88921923;+46139861;-67044156;E4Q9k
+20639012;-20888969;+36031161;-38143786;drCC3 
Signature 

Brian Beharry, QA
Infobright

Profile
 
   
1 of 3
1