UBB.Developers Forums UBB.threads 7 Development How Do I? Help? Importing Posts from another Message Board

Hi

I have been referred here by JoshPet to you geniuses that know this kinda stuff ..I hope you can help.

I am slowly migrating my users over to UBB Threads from a free message board service(Boards2Go)

They don't have any archiving method whatsoever(the board holds max 300 messages) so the board host has been emailing me all the posts that have dropped off.
There are close to 3000 of them so far.

I want to create an archive section forum on my new UBBThreads forum, my question is is there any way I can import these messages into my Threads board?

All messages have been emailed to me so I can make them into a CSV file or some kind of a db format

I am not a programmer or that database savvy, but is there an SQL command I could use in the Admin area of threads to import these messages? Is there a program that can easily convert them to something UBB might like?

For your reference our old message board is here:
http://www.b2g3.com/boards/board.cgi?user=CYABC

And when they get sent to me after they drop off the board, the Subject line for all the emails says "CYABC ARchive"
and the contents of the email will look like this:

Post Subject: Re(1): Edmonton SPCA
Author IP:
Author Email:
Author: Sounds great to me. .
Date and Time: 5/27/2001 @ 11:24:41 AM
Message: This is a subject I dont know much about but are they any<BR>blanket studies of all the SPCA's across Canada?<BR><BR>Do they communicate with each other? Is there a National<BR>SPCA convention of some sorts?<BR><BR>Is Edmonton's Mission something they live by or is it just<BR>blah blah blah?<BR><BR>


Our new UBB board is here:
http://cyabc.infopop.cc/ubbthreads/ubbthreads.php

Thanks in advance and I hope I explained this accurately

Lerosia

Sorry Lerosia,

I figured someone would know if it's possible or not.

I know there are ways to convert other database type boards like vb and phpbb etc... but don't know about getting the text files into a database.

Maybe someone will come along who knows.

If all the emails could be saved into one long plain text file (or one for each mail) they could be quite easily parsed.

The main problem would be the insert, but it shouldn't be too hard to modify the addpost-script from threads.

One caveat is what to do with users, would it be enough if the posts are made by the anonymous but showing the correct name. This way no profiles would have to be done.

It can be done.

I'm just not sure how well I could do it. LoL I haven't done much with such things as loading from a CSV but someone else may have. I think Dave_L has mentioned such things. Perhaps he has some sort of import tool. Are ya listening Dave?

It's not hard to write a quick and dirty script to import a CSV file into a MySQL database. I could provide a simple example, if someone wants it.

As Gardener mentioned, the problem here is deciding what to do about the usernames and profiles. The simplest solution would be to assign all the posts to an existing user.

Another issue is whether the old posts are all separate, or whether they're "threaded". If the latter, the CVS file would need to contain the necessary info for preserving the threads.

The final issue is finding someone with time to actually do this.

hehe good points.

From the looks of it they are not threaded so that should make it simpler. I think I would assign them under usernumber 1 with the verying usernames so as to not affect anyone else's post counts. (provided such usernames are not being used already)

Is the main goal here just the ability to have them available for viewing in .threads?

Hi Josh

I was suprised it took so long but I knew someone who have some idea of what to do

Thanks for the follow-up post to edge it along

Will post mo info below

Lerosia

Hi all,

I am not really concerned with usernames and profiles because there were no profiles on the old board.

I can save them all as one CSV file

Some (most)emails have come to me(the board host send me the posts that drop off in email format)like this:
(its automated I assume)

--------------------------------------------------------


----- Original Message -----
From: <****@boards2go.com>
To: <>
Sent: Thursday, August 29, 2002 3:51 PM
Subject: Boards2Go.com - CYABC ARCHIVE


> Post Subject: Not no kill by having others do it
> Author IP:
> Author Email:
> Author: Scrutinizer
> Date and Time: 5/30/2001 @ 03:17:05 AM
> Message: What would compel a pound take an SPCA dog or cat? I am certain the pound is not going to provide the service for free. Sending your animals to another place to die is not no kill.<BR><BR>Donors are not happy. The SPCA sent pets to die at the hands of others. Once given to the pound, who knows what happens to the pet?<BR>
---------------------------------------------------------

And others we were attempting to archive ourselves and came like this:
(sorry for long email here but we were putting the whole thread itself into an email..very arduous process)

------------------------------------------------------------
----- Original Message -----
From: **********@aol.com
To:
Sent: Tuesday, August 13, 2002 11:25 PM
Subject: History Repeats Itself - "Austin"


History Repeats Itself - "Austin"
Posted on 7/12/2002 at 12:41:17 AM by Anita Horne

Your website is dedicated to the 6 nice dogs that were killed after months under the care of the SPCA.

I fear that this has happened again. Austin had been picked up by the Victoria SPCA with serious injury from a chain embedded in his neck. He was taken and hospitalized but survived and put up for adoption. He was described as a happy dog yet was killed 3 months later. I cannot get an explanation. A number of people were following his progress since he survived his injury and no one can get an answer why he was killed. No response to my letter. Only 3 weeks earlier he was at the Pet Fair in Victoria and available for adoption. Two assessments were done, yet an explanation has not been received.

Does anyone who reads this board know why Austin was killed? If something happened to him, medically or behaviourally, then surely it must have happened during the 3 MONTHS he was in care of the Victoria SPCA.

Or is this going to be another case where an animal is destroyed and supposed to be forgotten about?

Anita Horne
Re(1): History Repeats Itself -
Posted on 7/12/2002 at 10:07:01 PM by Anonymous

I hope Kimberly can research this for you.
------------------------------------------------------------

Etc etc

I wont really worry about the latter format above, I can fix those later on. But most of the emails are sent as individual posts like the first example.

I can easily assign the username as mine(admin)

I have imported all the messages into Outlook
(I usually use OE) and can convert to CSV easily.

Just need some kind of guideline to go by.

I want to attach the csv file to this message but probably not a good idea, but it might help if someone saw how it was preserved.


If all the above seems confusing I am happy to be coached via email if someone decides to take this task on

Thanks in advance

Lerosia

[][email protected][/]

Hmm the Jeopardy Theme song was playing a long time on this one

Theres tumbleweeds on this subject.

I guress there isnt an answer to this dilemma eh?

Lerosia

If the CSV file has 3000 messages, I guess it's pretty large. Could you attach a piece of it, for example, 50 messages?

As Dave_L says it would be good to see a portion of the csv file.

There will also be some problems with preserving the threads. It seems that there is no way to see which thread a post belongs to except by the subject in the first format. If there are threads with the same subject they will be merged into one. All replies in a thread will also be a reply to the first post of the thread after the import, but hopefully that won't be much of a problem.

In the second format there won't be a problem separating the threads since there is one thread per mail. On the other hand there might be some problems parsing since there is no identifying text in front of the subject or where message ends. But if no message has "Re(1):" (or any number, if this changes in every message) at the start of a line in them this shouldn't be a problem either.

Hi Dave and Gardener,

Sorry I didnt respond sooner.

I have attached a small CSV file with 24 messages in it and I have trimmed it

The only thing I want gone in the formulas is the Author IP
I jsut couldnt figure out how to do find andreplace *.* in a CSV file

Hope this helps

Lerosia

PS I couldnt upload a CSV to your board so I saved as a tab delimited text file

PS

Heres a link to how I am archiving these puppies manually..very tedious process

http://cyabc.infopop.cc/ubbthreads/postlist.php?Cat=&Board=archives

Just thought I'd let everyone know that I'm working on this. I have the parsing working, now I'll just have to do the part that actually adds the post to the database.

Gardener! You are 'da Man!!!!

I have a small question. What does the number in the Re(1): part of the subject mean? Is it the number of the post in the thread that this post is a reply to?

If it is, I wonder if it is important to keep the structure or if all replies in a thread can be added as a reply to the first post of the thread?

I have finished a script that reads the file you provided and imports all messages into the database. The script is attached to this post.

There are some things you have to think about when you use it though. First, HTML isn't allowed in usernames on UBB.threads, so posts containing usernames with HTML won't be added to the database. You could easily edit these in the file before you import the messages and remove the HTML yourself.
Another thing is that I don't think there is a way to keep the thread structure on replies where the subject has changed, so they will become threads of their own.

How to use the importer
1. Download the attached file and rename it to postimporter.php
2. Edit the variable $messagefile at the top and set it to the name of your message file. If it will be in the same directory you only need the name, otherwise you will need the complete path to it.
3. Edit the variable $BoardKeyword and set it to the keyword of the board that you want the messages to be inserted to.
4. If you don't want the IP of the posters to be in the imported messages you should set the variable $saveip to "".
5. Upload the file to the root directory of your UBB.threads install. Also make sure that the message file is in the correct place.
6. Run the script from your browser, or from the command line if you have a stand-alone php binary:
> php postimporter.php

Done! You should now have lots of new (old) posts in your UBB.threads.

Hi
The number does represent the firstreply to post in the thread Re:(1), Re:(2) etc

Its really not all that important that I keep those because some of them I will have to do manually anyways because the
users are constantly changing the subject line within the thread.

Absolutely anything you can do to help will be better than manually entering them in.

At the rate I am going it will take about 2 weeks to enter them all in

Thank you so much

Lerosia

PS

I second what JoshPet says You are DA MAN!

Thank you very much!

I am not going to attempt this until I have a fresh mind in the morning, this one might not be as easy as I think.

I will let you know what happens.

Lerosia

That sounds like a good idea, and it is a good idea to make a backup of your database before so that it is easy to remove the posts if something goes wrong.