import-big-csv-file

Questionsimport-big-csv-file
Georges asked 1 month ago

Hi! I need to import large CSV files (30 000 lines) It takes very long time (and web error 504) is it possible to import using begintransaction() and commit ?? OR using   function do_import($data) to write my own import routine, but How to read CSV file line by line ?   Thaks for your help   Regardd   Georges

3 Answers
Abu answered 1 month ago

Hello,

I reviewed the code and it seems like it’s due to the limitation of php memory limit.

The current implementation is to store the uploaded file in the php array then to session at step2. And on step3, reads the array from session and insert in database.

So most likely, it’s a php memory limit case. You can confirm it by checking php error log.

The current implementation can also be optimized. let me know if increasing memory limit does not solve your case.

Georges answered 1 month ago

Hi Abu

I wrote a specific PHP code to solve my problem, using begintransaction() commit()

but it’s not a long term solution because CSV file structure can change

I don’t know how to change php memory limit. Can you explain?

Regards

Georges answered 3 weeks ago

Hi Abu,

I tried on export xls file, and with higher memory limit it’s ok.

thanks

Georges

Your Answer

3 + 19 =

Login with your Social Id:

OR, enter

Attach code here and paste link in question.
Attach screenshot here and paste link in question.



How useful was this discussion?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate it.

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?