FlowHeater Demo Video
 
FlowHeater - The Data Import / Export Specialist
 
Welcome, Guest
Username: Password: Remember me
  • Page:
  • 1

TOPIC: Bug when dealing with very large files

Bug when dealing with very large files 1 month 3 weeks ago #3475

  • FlowHeater-Team
  • FlowHeater-Team's Avatar Topic Author
  • Offline
  • Posts: 310
Hello,

I find that when I use FlowHeater to transfer files with over 20,000,000 rows into SQL I always get an error message at the end:

Execution Timeout Expired: The timeout period elapsed prior to completion of the operation or the server is not responding

It appears that this message appears after the last row is inserted, so the operation is effectively finished, but instead of completing normally, FlowHeater generates this message..

Mike
Request received via Email

Best wishes
Robert Stark

Did this answer your question? We would be grateful if you provide a brief comment as feedback. It may also help others who may have encountered a similar problem.

Please Log in or Create an account to join the conversation.

Best wishes
Robert Stark

Did this answer your question? We would be grateful if you provide a brief comment as feedback. It may also help others who may have encountered a similar problem.

Bug when dealing with very large files 1 month 3 weeks ago #3476

  • FlowHeater-Team
  • FlowHeater-Team's Avatar Topic Author
  • Offline
  • Posts: 310
Hi Mike,

This error message is related to the FlowHeater standard settings for databases transaction. By default the database Adapters using one database transaction for the import process. This works for small data. For large data, the database server cannot handle such transaction in a certain timeframe and FlowHeater run into the defined command timeout!

You can change the default database transactions setting on the advanced tab into the adapter configurator.

In your case, you´ll get the best result with the following settings.
-> Use database transactions (YES)
-> Perform AutoCommit after 10.000 written records (YES)

Note: I do not recommend disabling database transaction because you will lose performance during the import process!

I guess these settings should solve your issue.


Best wishes
Robert Stark

Did this answer your question? We would be grateful if you provide a brief comment as feedback. It may also help others who may have encountered a similar problem.
Attachments:

Please Log in or Create an account to join the conversation.

Best wishes
Robert Stark

Did this answer your question? We would be grateful if you provide a brief comment as feedback. It may also help others who may have encountered a similar problem.

Bug when dealing with very large files 1 month 3 weeks ago #3477

Yes that solved the problem. Thank you, Robert.

Please Log in or Create an account to join the conversation.

  • Page:
  • 1

other Languages

en
GB | US

de
DE | AT | CH

FlowHeater Home

de en

Imprint/Contact

Privacy Statement

Copyright © 2009-2018 by FlowHeater GmbH. All rights reserved.

Follow us on

twitter  facebook

YouTube

 de en