Speed - by PhilipOrleans

Status : 

  Fixed<br /><br />
		This item has been fixed in the current or upcoming version of this product.<br /><br />
		A more detailed explanation for the resolution of this particular item may have been provided in the comments section.

Sign in
to vote
ID 3143418 Comments
Status Resolved Workarounds
Type Suggestion Repros 0
Opened 11/5/2017 7:09:26 AM
Access Restriction Public


I wonder when Microsoft will hire a database expert from Linux and make sure MS SQL can actually even compete.
I just exported to a *.csv file 110MM records, one 10 digit number each, from MySQL using RocksDB as engine, in 1.5 minutes. The same operation would take 1000% longer in any version of MS SQL. I can show you any time.
Sign in to post a comment.
Posted by PhilipOrleans on 12/4/2017 at 8:58 AM
This is called Big Data. I handle Call Detail Records for a large telecom company and typically I upload 700 to a BN records every day. Using free MariaDB is way easier and way, way faster. Microsoft could work on matching the usability and speed of those applications, or slowly lose the market to those free databases until MS SQL would be irrelevant. For example the sql statement insert, or replace. It is way easier to use than Merge. For bulk insert I could care less if the data is already there, if it is, ignore the insert, if it is not, insert it. The only way to do this with MS SQL is to create an "instead of" trigger and a merge statement. Why work so much? bcp should take care of that. And so on. Microsoft needs compete with the new databases, not ignore them.MS SQL is going the way of Sybase.
Posted by Koen Verbeeck on 12/4/2017 at 3:16 AM
By the way, I exported 53 million rows from a SQL Server table to a flat file (also just numbers) in 35 seconds, or about 1.5 million rows per second. According to your test, you achieve a speed of about 1.2 million rows per second, so I'm not sure where you get the "1000% longer" from.
Posted by Koen Verbeeck on 12/4/2017 at 2:56 AM
Yes, the primary goal of any RDMS is to export 110 million rows, where each row just has one number...