Joinutility seperatorLogin utility separator Infobright.com
   
 
SELECT INTO OUTFILE / LOAD DATA INFILE time zone issue
Posted: 20 July 2010 04:14 PM   Ignore ]  
Newbie
Rank
Total Posts:  16
Joined  2009-11-09

Hello.

We are using ICE for data warehousing.
I need some help explaining this:
SELECT INTO OUTFILE on ICE seems to transform timestamp columns to UTC. But LOAD DATA INFILE… I cannot say what its supose to do.. It should convert from UTC to client time zone, but does not look to be doing it.

I will try to reproduce it with a simple example.

SET @bh_dataformat 'txt_variable';
SHOW GLOBAL VARIABLES like '%time_zone%';
+------------------+---------------+
Variable_name    Value         |
+------------------+---------------+
system_time_zone CEST          |
time_zone        Europe/Madrid |
+------------------+---------------+

SELECT now() INTO OUTFILE '/tmp/time.csv';
SELECT now();
+---------------------+
now()               |
+---------------------+
2010-07-20 21:13:08 |
+---------------------+

CREATE TABLE `timezoneTest` (
  `
col1timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
ENGINE=BRIGHTHOUSE

shell$ cat /tmp/time.csv
2010-07-20 21:13:04

Everything looks normal until here.

LOAD DATA INFILE '/tmp/time.csv' INTO TABLE timezoneTest;
SELECT FROM timezoneTest;
+---------------------+
col1                |
+---------------------+
2010-07-20 22:13:04 |
+---------------------+ 

Ok, why +1 hour?

And now, the strangest thing:

SELECT FROM timezoneTest INTO OUTFILE '/tmp/time_loadInfile.csv'

shell$ cat /tmp/time_loadInfile.csv
2010-07-20 20:13:04
(2 hours less, this could be reasonable. If infobright loader converts to UTC on SELECT INTO OUTFILE as Europe/Madrid is +2 on summer time)

LOAD DATA INFILE '/tmp/time_loadInfile.csv' INTO TABLE timezoneTest;

SELECT FROM timezoneTest;
+---------------------+
col1                |
+---------------------+
2010-07-20 22:13:04 |
2010-07-20 21:13:04 |
+---------------------+ 

Ok, why is happening this? I mean. If I SELECT INTO OUTFILE and then LOAD DATA INFILE, shouldn’t I obtain the same result?????

Obviously I missed something…

If this same test is done with @bh_dataformat = ‘mysql’, then behaviour is as expected, SELECT INTO OUTFILE + LOAD DATA INFILE returns the same timestamp than original from table.


Thanks a lot for the future answers! I’m just a bit stucked… wink

Profile
 
Posted: 21 July 2010 05:08 AM   Ignore ]   [ # 1 ]  
Super Duper Member
Avatar
RankRankRankRankRank
Total Posts:  916
Joined  2008-08-18

Hi !

What ICE version do you use?
I checked the coming 3.4.2 (to be released very soon) and the situation looks like that:

1.

LOAD DATA INFILE '/tmp/time.csv' INTO TABLE timezoneTest;
SELECT FROM timezoneTest;
+---------------------+
col1                |
+---------------------+
2010-07-20 22:13:04 |
+---------------------+ 

Ok, why +1 hour?

This works fine now, the time loaded into a table is same as in the load file

2.

SELECT FROM timezoneTest INTO OUTFILE '/tmp/time_loadInfile.csv'

shell$ cat /tmp/time_loadInfile.csv
2010-07-20 20:13:04

This works as you have guessed, Infobright loader converts to UTC on SELECT INTO OUTFILE. This behavior leads to problems, as exported data are converted to UTC, but loaded data are treated as local timezone. So “20:13:04” is exported as UTC, but at load it is treated as Europe/Madrid

I guess you can create a defect for this in http://bugs.infobright.org/

Profile
 
Posted: 21 July 2010 05:14 AM   Ignore ]   [ # 2 ]  
Newbie
Rank
Total Posts:  16
Joined  2009-11-09

Thanks a lot for the quick reply Janusz.

We are using ICE 3.3.1
From mysql at login: Server version: 5.1.40-log build number (revision)=IB_3.3.1_r6997_7017(ice).

I will post on bug list the situation described above right now.

Profile
 
Posted: 26 July 2010 09:18 PM   Ignore ]   [ # 3 ]  
Jr. Member
Avatar
RankRank
Total Posts:  59
Joined  2009-05-29

The MySQL 5.0 Manual, at http://dev.mysql.com/doc/refman/5.0/en/timestamp.html, says, “TIMESTAMP values are converted from the current time zone to UTC for storage, and converted back from UTC to the current time zone for retrieval.”

If “SELECT INTO OUTFILE” is considered “retrieval”, the TIMESTAMP values selected into the outfile should be converted from UTC to the current time zone.
Since “LOAD DATA INFILE” is storing data, the TIMESTAMP values stored should be converted from the current time zone to UTC.
Thus, if SELECT INTO OUTFILE and LOAD DATA INFILE are run in sessions in the same time zone, the data in a TIMESTAMP column should be unchanged by that processing.

Profile