ibneko: (Default)
I've been trying to convert a phpBB2 forum to a phpBB3 forum.

One of the biggest problems I keep running into is that posts get lost at a certain point. I'm not entirely sure why, or how.

But the issue I'm going to address here is that upgrading a LARGE forum database (ours is at 3 GB, with around 4 million posts) was really slow, especially at step 16 of 28, filling the phpbb_posts table.

Why? To ensure compatibility with other converters, other databases, different designs, the phpBB developers chose to use a SELECT query with LIMIT x, y. Where x = number of rows at which you want to start querying, and y = number of rows to return.

When x gets really big, the speed at which queries will come back will get really slow, in the tens of seconds range.

So instead, I hacked install_convert.php so that when it starts processing phpbb_posts, it'll disregard the skip_rows parameter (which establishes 'x') and instead set x to 0, while adding to the WHERE part of the query a condition stating that we only want to select posts with post_id greater than the largest post_id we have in our new phpBB3 phpbb_posts table.

Here's a diff.
Read more... )

Mind, I'm still running this right now, so I don't know if it definitely works*, but estimated runtime for my posts table is down to 4 hours and it's holding a steady 330/s row processing rate (as opposed to starting at 330/s and dropping down to less than 100/s row and taking over 14 hours).

*as in, there may be errors down the road?

[edit] As far as I could tell, this worked perfectly.
So if the issue you're experiencing with upgrading phpBB 2.0 to phpBB 3.0 is due to the fact that the upgrade process is too slow at step 17 of 28 (phpbb_posts table), this patch should be safe to use. It will decrease processing time by a significant amount: I think processing step 17 was cut down from 14-16 hours down to a bit over 4 hours.
ibneko: (Default)
....I think they're the dumbest thing ever. Along with the lack of initializing variables.

I just spent over an hour trying to figure out why:
$userid = -1;
was overwriting the value in $_SESSION['userid'].

::really disliking PHP now::

WTF...

Feb. 14th, 2007 04:31 am
ibneko: (Default)
Why does my server not like:

if (isset($_POST['url'])) {
#clean URL
print $_POST['url'];

    /*
$out = escapeshellarg("addblog-test.pl $url");
print $out;
*/
}

if ($errors){
print "
$errors
";
}

(it gives me an error:
Parse error: syntax error, unexpected '}' in /home/uvemympn/public_html/dev/addblog.php on line 16, which line 16 is the last line.)

Yet it's perfectly fine if I take out the comments?
if (isset($_POST['url'])) {
#clean URL
print $_POST['url'];
}

if ($errors){
print "
$errors
";
}


I don't get it. I really don't. I even tried different ways of commenting... # and // both...

CSS, part 3

Jan. 6th, 2006 11:51 pm
ibneko: (Default)
So the conclusion I arrived at?

Leave most of the work intact. The icky parts that look better in tables are displayed as tables by default. However, if you want CSS, a simple ?style=CSS will enable CSS for you.

It's probably a waste of space, but this means that if, before hell freezes over, someone actually manages to come up with browsers that support CSS correctly all around, it'll look perfect.

--
Right now, my focus is on patching any and all security holes and converting the coldfusion stuff to PHP. Most of it's easy, to the point where I could probably write up a perl script to parse coldfusion, write it out to PHP, and be done with it. But the problem with that is, stuff's going to get missed, and then I'll have to track it down later. But CSS pretty-fying will have to wait. I've got a week left before all hell breaks loose again.

Le sigh. I'm thinking of resigning my position. This probably looks pretty good on my transcript, but... I dunno. I'm not going to have the time once school starts, to make sure everything runs right and stuff. I've promised myself that I would try (even harder, yeah, I know I said this before. I know.) for straight A's. That it's possible. Even for someone like me, who sucks at taking exams, and is insanely careless when it comes to... just about everything. 'cept coding, which I can play perfectionist at, if I understand the syntax and language.

Heck, I'm not even sure I'll be able to keep the job at T.I.S. Brian (Bryan?) said he'd welcome me back, but... I dunno. I'm afraid I might be giving too much of my time to non-school things.. that that's why I'm failing.

--
On the brighter side, senpai told me to stop worrying. Posting that locked, so I don't forget. =^^= Senpai, I love you. Even if you probably don't read this.

Expand Cut Tags

No cut tags

Profile

ibneko: (Default)
ibneko

Syndicate

RSS Atom

Most Popular Tags

Style Credit

Page generated Oct. 18th, 2017 05:39 am
Powered by Dreamwidth Studios
November 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 2016