Infobright.org Forums

Infobright DomainExpert error log

2012-02-07T04:10:09-04:00

I need some help.My English is poor.
ICE_User_Guide document”
A value inserted into a column with a decomposition defined does not have to match the rule. Such non-matching values are inserted into a separate subcollection. This subcollection of outliers is compressed and stored independently of other subcollections.
You can obtain the accuracy of decomposition rules by setting the ControlMessages parameter in the brighthouse.ini file to 2 (or higher):ControlMessages = 2
If the parameter is set on each COMMIT for each column, Infobright reports the number of outliers among the committed values (from INSERTs and LOADs).For example:
2011-05-25 16:59:03 Decomposition of ./network/connection.ip left 15 outliers.
“

I set ControlMessages = 4,but I didn’t find in bh.err

mysql> show variables like 'version_comment';
+-----------------+--------------------------------------------------+
| Variable_name   | Value                                            |
+-----------------+--------------------------------------------------+
| version_comment | build number (revision)=IB_4.0_r13151_13690(ice) |
+-----------------+--------------------------------------------------+

mysql> show variables like 'brighthouse_ini_controlmessages';
+---------------------------------+-------+
| Variable_name                   | Value |
+---------------------------------+-------+
| brighthouse_ini_controlmessages | 4     |
+---------------------------------+-------+

mysql> desc t8;
+-------+----------+------+-----+---------+-------+
| Field | Type     | Null | Key | Default | Extra |
+-------+----------+------+-----+---------+-------+
| n     | char(10) | YES  |     | NULL    |       |
+-------+----------+------+-----+---------+-------+
1 row in set (0.00 sec)

mysql> CALL sys_infobright.SHOW_DECOMPOSITION('test','t8');                 
+---+-------------+-----------+---------------+--------+
| # | column name | data type | decomposition | rule   |
+---+-------------+-----------+---------------+--------+
| 1 | n           | char      | t8r           | :::.%d |
+---+-------------+-----------+---------------+--------+

mysql> begin;
Query OK, 0 rows affected (0.00 sec)

mysql> load data infile '/tmp/b.txt' into table t8;
Query OK, 1 row affected (0.03 sec)
Records: 1  Deleted: 0  Skipped: 0  Warnings: 0

mysql> commit;
Query OK, 0 rows affected (0.00 sec)



[root@test /tmp]# cat b.txt
"abcde" 

rule is :::.%d
“abcde” is non-matching values
but I didn’t find in bh.err,why ?

Compilation issue in Infobright 4.0.3 with boost 1.42.0

2012-02-07T04:02:33-04:00

Hi,

I tried compiling infobright. I’ve installed all the required dependencies along with boost 1.42.0. So when I execute the command
make EDITION=community release, I get the following error.

checking whether the Boost::Program_Options library is available... yes
checking for exit in -lboost_program_options... no
checking for exit in -lboost_program_options... (cached) no
checking for exit in -lboost_program_options... (cached) no
configure: error: Could not link against boost_program_options !
make: *** [build/community/release/infobright/Makefile] Error 1 

After examining the config.log file in infobright-4.0.3/build/community/release/infobright
I find the following error.

configure:24792: gcc -o conftest  -O3  -finline-limit=32000     -D_THREAD_SAFE -D_REENTRANT -D_POSIX_PTHREAD_SEMANTICS   -m64 -fprefetch-loop-arrays   -fno-strict-aliasing -fno-merge-constants -fno-gcse   -DNDEBUG   -I/usr/local/boost_1_42_0/include  -L/usr/local/boost_1_42_0/lib conftest.c -lboost_program_options  -lcrypt -ldl -lcrypt -lnsl -lm  >&5
conftest.c:54: warning: conflicting types for built-in function 'exit'
/usr/bin/ld: skipping incompatible /usr/local/boost_1_42_0/lib/libboost_program_options.so when searching for -lboost_program_options
/usr/bin/ld: skipping incompatible /usr/local/boost_1_42_0/lib/libboost_program_options.a when searching for -lboost_program_options
/usr/bin/ld: cannot find -lboost_program_options
collect2: ld returned 1 exit status 

Could you please help? Has this error been encountered before?

Question on ICM

2011-09-10T06:04:59-04:00

Hi ALL
I use the ICM to detect data block corruption, found that the first run takes longer,
after the test results are instantaneous output.
Does a shorter time-consuming will it affect the the correctness of test results?
Why would the first time took longer?

And then,I use UEdit to modify the data file, resulting in data corruption.
Does IB have such tools similar with dbv or DBMS_REPAIR in oracle can handle it?

Problem Loading Date data using Load Data

2010-11-06T06:46:48-04:00

I am trying to load data into a table which has a datetime field

The data in the csv is delimited using semicolon (;) below is the details of What I am doing

—Create Table
CREATE TABLE CODE_B_DIM(
CODE_B_KEY BIGINT NOT NULL
,CODE_B VARCHAR(10)
,CODE_B_DESCRIP VARCHAR(40)
,CODE_B_GROUP VARCHAR(40)
,CODE_B_SUB_GROUP VARCHAR(40)
,VALID_FROM DATETIME
,VALID_UNTIL DATETIME
,SORT_VALUE VARCHAR(30)
,CODE_B_GROUP_KEY BIGINT
,CODE_B_SUB_GROUP_KEY BIGINT
) engine=brighthouse;

—Load Data into infobright using the csv file having semicolon as delimiter.
————————————————————————————————————————————
LOAD DATA INFILE ‘D:\temp\code_b_dim.txt’ INTO TABLE CODE_B_DIM FIELDS TERMINATED BY ‘;’;

Format of the CSV file
————————————————————————————————————————————-
1;C0010;PRODUCTION;PRODUCTION;PRODUCTION;01/01/1980;31/12/2049;C0010;;
2;C0011;PROCESS/RECOMBINING;PRODUCTION;PRODUCTION;01/01/1980;31/12/2049;C0011;;

The Date fields are not being loaded properly into the database. The date format is DD/MM/YYYY
Date field Value in the database is
0000-00-00 00:00:00

My OS is Windows.

All the other fields data is fine.

Thanks & Regards

Ishaq

Infobright crash and automatic restart

2012-01-02T06:23:36-04:00

Following is the error log written in bh.err file
################################

Note ::- Over infobright version is :: Infobright Enterprise Edition 3.4.1 (IB_3.4.1_r8746_8921(iee - commercial))

##### START OF LOG OF INFOBRIGHT CRASH AND RESTART ON 111228 23:36:07 ####

111228 23:36:07 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=16777216
read_buffer_size=1048576
max_used_connections=10
max_threads=151
threads_connected=5
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 327165 K
bytes of memory
Hope that’s ok; if not, decrease some variables in the equation.

thd: 0x2b995ff3f6b0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong…
stack_bottom = 0x2b9948a83f50 thread_stack 0x80000
[0x9a7499]
[0x4027ea]
[0xaea9f0]
[0x830337]
[0x698ad9]
[0x690ee5]
[0x6919c8]
[0x680e6a]
[0x68b7c9]
[0x68a1b4]
[0x68ac83]
[0x80ece6]
[0x6c288a]
[0x5951da]
[0x595538]
[0x4edbf4]
[0x5f5a38]
[0x5f672c]
[0x5f3c4b]
[0x5f42fb]
[0x4b7470]
[0x5f14fc]
[0x5f5a00]
[0x5f672c]
[0x5f3c4b]
[0x5f42fb]
[0x4b7470]
[0x4ba730]
[0x4bb6f6]
[0x4bbe83]
[0x4ae5b1]
[0xae5c0d]
[0xb2b2d9]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort…
thd->query at 0x2b99626eb360 is an invalid pointer
thd->thread_id=107469
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

111228 23:36:09 mysqld_safe Number of processes running now: 0
111228 23:36:09 mysqld_safe mysqld restarted
Loading configuration for Infobright instance ...
Option: AllowMySQLQueryPath, value: 1.
Option: AutoConfigure, value: 0.
Option: CacheFolder, value: /usr/local/infobright-3.4.1-x86_64/cache.
Option: ControlMessages, value: 0.
Option: InternalMessages, value: 0.
Option: KNFolder, value: BH_RSI_Repository.
Option: KNLevel, value: 99.
Option: LicenseFile, value: <unknown>.
Option: LoaderMainHeapSize, value: 800.
Option: PushDown, value: 1.
Option: ServerMainHeapSize, value: 10000.
Option: UseMySQLImportExportDefaults, value: 0.
Infobright instance configuration loaded.
111228 23:36:09 [Note] Event Scheduler: Loaded 0 events
111228 23:36:09 [Note] Infobright Enterprise Edition 3.4.1 is ready for connections.
socket: ‘/tmp/mysql-ib.sock’ port: 5029 build number (revision)=IB_3.4.1_r8746_8921(iee - commercial)

##### END OF LOG OF INFOBRIGHT CRASH AND RESTART ON 111228 23:36:07 ####

Remote/Silent Windows Installation

2012-02-01T17:06:55-04:00

Is it possible to install Infobright on Windows using the command line or a .bat file?

DB Corruption

2011-11-21T11:33:45-04:00

We have a big problem. During our db import(load data in file) we encountered a power loss and data is no longer available. Is there a way to restore the data somehow? Data is there physically on the file system(26Gb total) but we cannot access it. This is the error we are getting:
mysql client:
ERROR 2013 (HY000): Lost connection to mysql server during query

...This is from brighthouse.log
Error in attribute file: bad format

...This is from bh.err…
111121 14:23:03 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=16777216
read_buffer_size=1048576
max_used_connections=1
max_threads=151
threads_connected=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 327165 K
bytes of memory
Hope that’s ok; if not, decrease some variables in the equation.

thd: 0x14e8290
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong…
stack_bottom = 0x7fe18aeede88 thread_stack 0x80000
/usr/local/infobright/libexec/mysqld() [0x92e499]
/usr/local/infobright/libexec/mysqld() [0x4181cd]
/lib64/libpthread.so.0(+0xf470) [0x7fe1b6d04470]
/lib64/libc.so.6(+0x7d0e2) [0x7fe1b6a0c0e2]
/usr/local/infobright/libexec/mysqld() [0x7e0a2d]
/usr/local/infobright/libexec/mysqld() [0x7ed198]
/usr/local/infobright/libexec/mysqld() [0x680ebc]
/usr/local/infobright/libexec/mysqld() [0x67f15a]
/usr/local/infobright/libexec/mysqld() [0x686435]
/usr/local/infobright/libexec/mysqld() [0x660305]
/usr/local/infobright/libexec/mysqld() [0x7363d7]
/usr/local/infobright/libexec/mysqld() [0x6b6f2d]
/usr/local/infobright/libexec/mysqld() [0x6c9dc7]
/usr/local/infobright/libexec/mysqld() [0x5a38dd]
/usr/local/infobright/libexec/mysqld() [0x50a4cd]
/usr/local/infobright/libexec/mysqld() [0x50455e]
/usr/local/infobright/libexec/mysqld() [0x5055b3]
/usr/local/infobright/libexec/mysqld() [0x506146]
/usr/local/infobright/libexec/mysqld() [0x506610]
/usr/local/infobright/libexec/mysqld() [0x4c55d8]
/usr/local/infobright/libexec/mysqld() [0x4ca9fb]
/usr/local/infobright/libexec/mysqld() [0x4ce78d]
/usr/local/infobright/libexec/mysqld() [0x4cf637]
/usr/local/infobright/libexec/mysqld() [0x4cff15]
/usr/local/infobright/libexec/mysqld() [0x4c248d]
/lib64/libpthread.so.0(+0x6c1a) [0x7fe1b6cfbc1a]
/lib64/libc.so.6(clone+0x6d) [0x7fe1b6a612cd]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort…
thd->query at 0x153bd40 = select * from sms limit 1
thd->thread_id=1
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
111121 14:23:03 mysqld_safe Number of processes running now: 0
111121 14:23:03 mysqld_safe mysqld restarted
Loading configuration for Infobright instance ...
Option: AllowMySQLQueryPath, value: 0.
Option: AutoConfigure, value: 0.
Option: CacheFolder, value: /usr/local/infobright/cache.
Option: ControlMessages, value: 0.
Option: InternalMessages, value: 0.
Option: KNFolder, value: BH_RSI_Repository.
Option: KNLevel, value: 99.
Option: LicenseFile, value: <unknown>.
Option: LoaderMainHeapSize, value: 320.
Option: PushDown, value: 1.
Option: ServerMainHeapSize, value: 600.
Option: UseMySQLImportExportDefaults, value: 0.
Infobright instance configuration loaded.
111121 14:23:03 [Note] Event Scheduler: Loaded 0 events
111121 14:23:03 [Note] Infobright Community Edition 3.4.2 is ready for connections.
socket: ‘/tmp/mysql-ib.sock’ port: 5029 Source distribution

...It seems unreasonable for the data to just become completely unavailable since the corruption should only affect that last “load data” batch. Complete data loss is not really acceptable.

Thanks for your help,
Damir

General Design

2012-01-21T01:11:22-04:00

Just testing things really but would appreciate any feedback/thoughts.

In a table with say 10 million rows and 800 columns, is it best to try and reduce the number of columns if possible or is it fine to leave with that many.

To give an example, say three of the columns use tinyid data type and could be combined into one smallint column with no real hassles.

Obviously with the existing set-up, these columns can be queried at any stage individually or collectively, however it would also be easy to merge them and query with a longer IN statement.

I guess what Im asking is do you believe it is better to reduce the number of columns where easily done in favour of having longer IN statements in the query? Or is it best to have the data as collapsed as possible?

As an example, which (if any) is the recommended design

(collapsed qry) WHERE ID1=16 AND ID2=33 and ID3=7
(columns merged qry) WHERE ID IN (8,19,26,67,89,90,92,98,99,108,189 etc etc etc)

Appreciate any thoughts you may have

Thanks

Chris

Using LOAD DATA LOCAL INFILE

2011-12-13T13:17:07-04:00

Hi,

I’ve been trying to use LOAD DATA LOCAL INFILE but I have been receiving ‘Unknown error’ every time.
I’m trying to load data into a remote server which is not in the same local network.
Since I was able to load data remotely using “LOCAL” keyword some time ago, I couldn’t understand what was going on now.
After many different tests I tried connecting to a server in the same local network and than it worked.
So there’s seem to be a problem when the server is not in the same local area, probably due to timeouts or connection speed.
Any clues or suggestions?
Maybe changing some configuration on the server or client can help?

Best regards,

Luis Fernando

Problem with GROUP BY and ORDER BY performance

2011-12-29T13:43:32-04:00

Hi,

I just installed InfoBright (4.0.5 on Ubuntu 10.04) to test its performance in my reporting application.

I have flat table with denormalized data about on-line ad impressions. Single row represents details about one ad impression (like user id, visited page domain, time, ad unit reference, campaign name, brand name etc). There is about 1 million rows in test database.

My basic report needs data from that table grouped by CAMPAIGN_ID (with counted AD_IMPRESSION_ID which represents number of impressions of ad units that belong to each campaign etc).

When I do select with just ‘ORDER BY’ or ‘GROUP BY’ clausule alone - query executes in about 0.1 second. But when I do select with both ‘GROUP BY ... ORDER BY ...’ query executes in about 20 seconds, and since it’s on-line application it is a problem.

Do you have any hints what should I check to improve performance of my query?

I’m attaching table definition and select query.

Regards,
palooh

———————————————————————————

delimiter $$

CREATE TABLE `BRIGHT_VIEW` (
`AU_ID` int(11) NOT NULL DEFAULT ‘0’,
`AU_CONTENT_TYPE` varchar(100) DEFAULT NULL,
`AU_CONTENT_BINARY_EXTID` char(15) DEFAULT NULL,
`AU_SIZE` varchar(10) DEFAULT NULL,
`AD_IMPRESSION_ID` int(11) NOT NULL DEFAULT ‘0’,
`AU_PLACEMENT` tinyint(4) DEFAULT NULL,
`AU_VISIBILITY` tinyint(4) DEFAULT NULL,
`CAMPAIGN_ID` int(11) NOT NULL DEFAULT ‘0’,
`CAMPAIGN_SAMPLE_LANDING_PAGE` varchar(2000) DEFAULT NULL,
`CAMPAIGN_NAME` varchar(100) DEFAULT NULL,
`CAMPAIGN_CONTENT` varchar(4000) DEFAULT NULL,
`BRAND_ID` int(11) NOT NULL DEFAULT ‘0’,
`BRAND_NAME` varchar(100) DEFAULT NULL,
`PAGE_VIEW_ID` int(11) NOT NULL DEFAULT ‘0’,
`IS_SUBPAGE` tinyint(1) DEFAULT NULL,
`VIRTUAL_USER_ID` int(11) DEFAULT NULL,
`PAGE_VIEW_TIME` datetime DEFAULT NULL,
`PAGE_DOMAIN_ID` int(11) NOT NULL DEFAULT ‘0’,
`PAGE_VIEW_DOMAIN` varchar(256) DEFAULT NULL,
`PAGE_VIEW_DOMAIN_TOP` varchar(256) DEFAULT NULL,
`IMPRESSION_LANDING_PAGE_ID` int(11) NOT NULL DEFAULT ‘0’,
`IMPRESSION_LANDING_PAGE` varchar(2000) DEFAULT NULL,
`SITE_ID` int(11) NOT NULL DEFAULT ‘0’,
`PUBLISHER` varchar(50) DEFAULT NULL
) ENGINE=BRIGHTHOUSE DEFAULT CHARSET=utf8$$

—————————————————————

select
CAMPAIGN_ID, CAMPAIGN_NAME, CAMPAIGN_CONTENT,
BRAND_ID, BRAND_NAME, CAMPAIGN_SAMPLE_LANDING_PAGE,
AD_IMPRESSION_ID, PAGE_VIEW_DOMAIN, PUBLISHER, PAGE_VIEW_TIME,
count(AD_IMPRESSION_ID) as IMPRESSIONS,
min(PAGE_VIEW_TIME) as FIRST_SEEN,
max(PAGE_VIEW_TIME) as LAST_SEEN,
count(distinct VIRTUAL_USER_ID)
from
BRIGHT_VIEW
group by
CAMPAIGN_ID
order by
CAMPAIGN_ID asc
limit 10 offset 0