Home Dashboard Directory Help
Search

SQOOP Export Command – HBASE and ZOOKEEPER are not found/missing? by vishal0soni


Status: 

Resolved
 as By Design Help for as By Design


1
0
Sign in
to vote
Type: Bug
ID: 783866
Opened: 4/16/2013 9:55:38 PM
Access Restriction: Public
1
Workaround(s)
view
0
User(s) can reproduce this bug

Description

We were trying to export a .dat file to the Windows Azure SQL DB using SQOOP Export command with the complete syntax as shown in the image posted at this link: http://social.msdn.microsoft.com/Forums/en-US/windowsazuredata/thread/62eb0422-3d9d-48d1-96c8-a4da2d25b877

But this commands stops every time at MAP 0% and Reduce 0% with the warning messages as shown in the image. It says that HBASE and ZOOKEEPER are missing and import will fail.

After getting this error, we tried to download the HBASE and ZOOKEEPER files from github and pasted them in the Hadoop directory. Also we tried to set the class path/environment variables to these extracted directories respectively. Is it the correct way of installing/using HBASE and ZOOKEEPER or is there any other way to perform this operation?
Details
Sign in to post a comment.
Posted by Microsoft on 6/23/2013 at 12:50 PM
I have tried to export an imported table or a .txt file from hdfs to a table in SQL server without a primary key, and it works. What happen if you don't terminate the mapreduce job after it hang? Can you attache the hadoop.log in /hadoop-1.1.0-SNAPSHOT/logs? If it is convenience, can you also attache the file you try to export and the table schema so we can reproduce the exact problem you encontered. As you mentioned, there is no error message to guide the user to solve the problem, so the imformation you provide is important for us to improve the logging of this kind of situation. Thanks
Posted by vishal0soni on 6/21/2013 at 3:50 AM
Ok, we got it working now. The problem was with the SQL DB Table structure only, the primary keys were not defined. But this was not reflected in the error message.

Thanks to Bill Ramos for providing the guidelines.
Posted by Bill Ramos on 6/20/2013 at 2:17 PM
It turns out that in likelihood, it's not having a PRIMARY KEY defined for the table for the SQL Database on Windows Azure.
Posted by Microsoft on 5/13/2013 at 7:31 PM
The HBase and Zookeeper not found message is just a warning, so unless your operation does not involve HBase (which I can already see), then this warning is harmless.

Regarding the error you are seeing, I am suspecting the problem might be in the delimiter format. Could you please try using a different format and let me know if the issue still persists. E.g. --input-fields-termiated-by \0x7C or ','

Also how many records do you have in the file to export. These was a known issue for exporting empty files to MS SQL/SQL Azure, which has been resolved.

Is it possible to provide logs if any
Sign in to post a workaround.
Posted by Bill Ramos on 6/20/2013 at 2:17 PM
It's unfortunate that you didn't include the CREATE TABLE statement, but it turns out you most likely forgot to include the PRIMARY KEY against the table.

If you create your table like this:
CREATE TABLE [dbo].[te_census_info]
(
        [State] [nvarchar](30) NOT NULL,
        [County] [nvarchar](50) NOT NULL,
        [AgeGroup] [nvarchar](20) NOT NULL,
        [Total_Population] [nvarchar](8) NOT NULL
        PRIMARY KEY ([State], [County], [AgeGroup], [Total_Population])
)

Sqoop will be able to insert the data.
File Name Submitted By Submitted On File Size  
270959.png 4/16/2013 28 KB