ibneko: (Default)
I've been trying to convert a phpBB2 forum to a phpBB3 forum.

One of the biggest problems I keep running into is that posts get lost at a certain point. I'm not entirely sure why, or how.

But the issue I'm going to address here is that upgrading a LARGE forum database (ours is at 3 GB, with around 4 million posts) was really slow, especially at step 16 of 28, filling the phpbb_posts table.

Why? To ensure compatibility with other converters, other databases, different designs, the phpBB developers chose to use a SELECT query with LIMIT x, y. Where x = number of rows at which you want to start querying, and y = number of rows to return.

When x gets really big, the speed at which queries will come back will get really slow, in the tens of seconds range.

So instead, I hacked install_convert.php so that when it starts processing phpbb_posts, it'll disregard the skip_rows parameter (which establishes 'x') and instead set x to 0, while adding to the WHERE part of the query a condition stating that we only want to select posts with post_id greater than the largest post_id we have in our new phpBB3 phpbb_posts table.

Here's a diff.
Read more... )

Mind, I'm still running this right now, so I don't know if it definitely works*, but estimated runtime for my posts table is down to 4 hours and it's holding a steady 330/s row processing rate (as opposed to starting at 330/s and dropping down to less than 100/s row and taking over 14 hours).

*as in, there may be errors down the road?

[edit] As far as I could tell, this worked perfectly.
So if the issue you're experiencing with upgrading phpBB 2.0 to phpBB 3.0 is due to the fact that the upgrade process is too slow at step 17 of 28 (phpbb_posts table), this patch should be safe to use. It will decrease processing time by a significant amount: I think processing step 17 was cut down from 14-16 hours down to a bit over 4 hours.

Expand Cut Tags

No cut tags


ibneko: (Default)


RSS Atom

Most Popular Tags

Style Credit

Page generated Oct. 18th, 2017 05:39 am
Powered by Dreamwidth Studios
November 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 2016