Please refrain from posting animated GIFs, memes, joke videos and so on in discussions other than those in the off topic area.

Dismiss this message to confirm your acceptance of this additional forum term of use.
You must be 16 or over to participate in the Brickset Forum. Please read the announcements and rules before you join.

Importing set list from Peeron

georgebjonesgeorgebjones Member Posts: 224
edited May 2011 in Brickset.com
Not too sure where to put this exactly, but I saw the option to import a set list from Lugnet. I didn't use lugnet, but I did use Peeron. It has an option to export to a text file in a couple different formats. Most of them I tried to import and Brickset said they weren't text files. I finally got the closest thing to an error when it told me there was no valid data in the file. I am fairly decent at computers what I am hoping is one of 2 things :)
1) Could Brickset be extended to import inventory from Peeron? I could send the Peeron formatted text file to whomever @Bluemoose, @Huw, I don't know for sure.
2) Could someone send me a lugnet formatted text file, so I could convert my Peeron one to a format that would work here? It is worth it to me to go through that trouble, since I think it would be less trouble than inputting each set one by one. My collection is nothing compared to many Brickset users, but still large enough that I don't want to have to reinput it. :) Thanks in advance.
«1

Comments

  • georgebjonesgeorgebjones Member Posts: 224
    The exact error is "The file does not contain any valid data lines"
    I see the option to export to peeron, and CSV, Tab-delimited, etc. When exported from Peeron, the first couple rows look like this:

    Quantity,Set Number,Set Name,Theme
    1,10026-1,SE Naboo Starfighter,LEGO / SYSTEM / Star Wars / Ultimate Collector Series
    1,4475-1,Jabba's Message,LEGO / SYSTEM / Star Wars / Classic

    I tried using Tab delimited and then changed it to CSV to see if that worked, and no, it did not.

    I assume you are looking for different column headers.
  • rocaorocao Administrator Posts: 4,274
    edited May 2011
    I'll email you my Lugnet formatted file. Are you... like... the only one using peeron for set lists? :P
  • MatthewMatthew Cheshire, UKAdministrator Posts: 3,715
    I used peeron before it died, well in effect it's still not dead, it's struggling through. (The (not very) new pig element was added this January!)
    madforLEGO
  • georgebjonesgeorgebjones Member Posts: 224
    @rocao, I probably am. I haven't updated it in a long long time. I used it in the before time. :-) Thanks.
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 5,804
    Send it to me, George (huw@brickset.com) and I'll take a look, although it's not likely to be any time in the next week or so.
  • rocaorocao Administrator Posts: 4,274
    ^ If it's two weeks out, you're likely to still get it functional before george manually tabs his list... haha
  • drdavewatforddrdavewatford Hertfordshire, UKAdministrator Posts: 6,002
    I still update my setlist in Peeron, although newer sets are often not in the Peeron database. Force of habit, I guess !
  • georgebjonesgeorgebjones Member Posts: 224
    Ok, I got it to work, but it gave me an ominous warning before I imported. It said my existing collection data will be overwritten. Does that mean that all of my sets will be deleted and replaced with the list in the file, or does it mean that if there is a set I have in brickset, it will be replaced with what is in the text file? Come to find out, I actually have more sets already in Brickset than in the Peeron dinosaur, so at that point it would be worth it to just re-enter the Peeron ones. Thanks for all the help. That's what I love about this community. :)
  • georgebjonesgeorgebjones Member Posts: 224
  • rocaorocao Administrator Posts: 4,274
    ^ hah, I was only basing that off your email where you claimed your nearly went into a profanity-laced tirade when you saw the format it was expecting :P
  • georgebjonesgeorgebjones Member Posts: 224
    ^ ok, fair enough :) hehe, it was a little deflating to see all that.
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 5,804
    The Lugnet import does delete your existing data, I felt that would be the best (and easiest) way to do it as most people would be coming from having it all recoded at Lugnet and nothing at Brickset. Actually it copies it somewhere first but it's not retrievable from the UI: I'd have to dig it out.
  • IstokgIstokg MichiganMember Posts: 1,893
    OK, since we're speaking of Peeron.... what is up with that? Did it go the way of (old) LUGNET when the owners had a split up and no one is manning the ship??

    I just now finally signed up for Facebook... and the irony is that I got requests (rather old I bet).... from Todd Lehman, Suzanne Rich (whatever her last name now is), Dan Boger and Jennifer (formerly Boger) Finch.

    I know that Admin Clark Stephens was adding items for the longest time to both sites... but what gives?

    I knew 2 years ago that there was trouble in Peeron when I tried adding old LEGO catalogs (shipped them the image)... they never got posted... and I sent an inquiry several times and never got a reply. I even asked Dan's ex Jennifer... she forwarded the message to Dan... still no reply.

    Rather sad really. And what is sadder about LUGNET is that Rene Hoffmeister took over... and we still have 2009 dates on the main page... no wait... there is a new 2011 date for a Canadian fan event... but it's like LUGNET and PEERON are slowly (over many years) dying.

    I bet that the only reason Peeron is still around is because any Bricklink sellers that show sales on Peeron still generate some income for the site.

    At least when poor Dan Jezek (owner of Bricklink) passed away last year, there were conttingency plans for it... but even there... does Dan J's mother own the site or Eric (can't remember his last name at the moment), the new "Admin"??

    So many questions....
  • drdavewatforddrdavewatford Hertfordshire, UKAdministrator Posts: 6,002
    Regarding Peeron, people still seem to be uploading set and parts data - I was shocked to find an inventory for a recent set the other day. It is however getting ever more out of date with every day that passes, which is a huge pity as I like the way it lists out the sets better than Bricklink (though there are some mistakes) and it has some unique features like the ability to log your loose pieces as well as your sets, and also the ability to tell you if you can build a set from your existing sets.

    I keep hoping that the site will be sold or donated to one or more motivated individuals who can help to drag it back from the brink, give it a shot in the arm and address some of the maddening performance issues.....
  • IstokgIstokg MichiganMember Posts: 1,893
    edited May 2011
    I agree drdave ... Apparently the automated features work... but no new LEGO catalogs have been added since 2007. I wish there was a complete central repository for all LEGO catalogs.
  • georgebjonesgeorgebjones Member Posts: 224
    In any event, I did enter all my inventory into Brickset instead of importing, although I did get the import to work. So if anyone hasn't entered theirs from Peeron into Brickset yet, there is hope! Thanks to @Rocao and @Huw:)
  • brickmaticbrickmatic Member Posts: 1,071
    edited May 2011
    OK, since we're speaking of Peeron.... what is up with that?
    My thoughts exactly. Someone needs to show that site some love.
  • davee123davee123 USAMember Posts: 791
    edited May 2011
    OK, since we're speaking of Peeron.... what is up with that? Did it go the way of (old) LUGNET when the owners had a split up and no one is manning the ship??
    Similar-- Peeron's admins and LUGNET's admins (as far as the set database) had a lot of crossover as it turns out. They're still maintaining both databases, although not all that speedily. Peeron's far more difficult since it involves inventories, and the UI for admins modifying inventories needs some work to be more friendly and easy to use.
    Suzanne Rich (whatever her last name now is),
    She's *my* wife now, as it turns out :)
    Jennifer (formerly Boger) Finch.
    Oddly enough, she got re-married to my freshman year college roomate!
    I know that Admin Clark Stephens was adding items for the longest time to both sites... but what gives?
    I think the owners of each site have been lax in terms of adding newer, active admins, or have had difficulty agreeing on them. And given that BrickLink and BrickSet offer more-or-less all the same features, there's less weight on their shoulders, since they're not as "core" as they used to be.
    At least when poor Dan Jezek (owner of Bricklink) passed away last year, there were conttingency plans for it... but even there... does Dan J's mother own the site or Eric (can't remember his last name at the moment), the new "Admin"??
    There were contingency plans for all the sites-- it's just that BrickLink's admins were more numerous and more active. As a selling forum, there was a very concrete reason for admins to be continually active, unlike Peeron or LUGNET, where action only needed to be taken when new sets came out or were discovered, and there were no real consequences of delayed action.

    As I believe, the most difficult part in each case is that nobody is intimately familiar with the code apart from the original author-- and in each case the original author has moved on in one way or another. There are some people that have ACCESS to the various codebases, but they don't really know it as well, and aren't very invested in it.

    DaveE
  • brickmaticbrickmatic Member Posts: 1,071
    edited May 2011
    @DaveE Thanks for the update. I would like to point out that if there is no one really interested in getting to know or invest in the codebase, then that would make the codebase a good candidate for being made open source. Seriously, it is so sad to see Peeron deteriorate as it has through neglect. Why not open up the entire resource to the community?
  • IstokgIstokg MichiganMember Posts: 1,893
    edited May 2011
    Thanks Dave E.... no disrespect about Suzanne... my comment was more based on the fact that I was too lazy to look it up... rather than any disrespect about her previous situations. I've always liked Suzanne (I know that's the name she preferred)... I could tell by her posts that she didn't want LUGNET to end up a shell of its' former self... but that's sadly what it is... just like I think that Jennifer Finch doesn't want Peeron to "rust", and sadly it's rusting slowly.

    BTW is Suzanne still on Facebook? ;-)

    Also... I'm a good acquaintance of Clark Stephens... who seems to be the guy trying to spend at least some spare time on the LUGNET and PEERON database.... a task that probably overwhelmes him.

    Much of Peeron parts inventory is in Brickllink... and much of their set inventory is in Bricklink, Brickset and parts elsewhere.

    But it's the catalogs that bother me. It seems that PEERON has a stagnant catalog database... we need to somehow get a complete catalog library out there somewhere. Bricklink doesn't store it.. and all the other sites only have a handful of catalogs of specific years.

    There are about a dozen AFOLs I know... that have an accummulation of about 200 LEGO catalogs... going back to day one. Somewhere there needs to be a central repository of LEGO catalogs. Peeron is not up to the task.

    And getting back to LUGNET... one doesn't get a warm fuzzy feeling about LUGNET when the front banner page still lists the major news as 2009 items.... mercifully the Canadians got a 2011 item added at the top. I like Rene Hoffmeister and all... but I bet it's next to impossible to manage 2 major websites... let alone 1....

    Cheers,
    Gary Istok

    P.S. Dave, tell Suzanne Gary says HI!!
  • IstokgIstokg MichiganMember Posts: 1,893
    LOL... Dave E.... I just went to LUGNET to check on a specific set... Samsonite LEGO Gear set #001... which John P. of Bricklink HAD 39 copies... but sold them all as well as the rest of his VAST collection (he admits to Obsessive Compulsive)... his comments are still in LUGNET (under set ownership).... so I have to EMAIL him and kiddingly harass him about removing a zillion set ownership comments he added to LUGNET long ago...

    ... and wouldn't you know it... the first item in LUGNET I've viewed in at least 2 weeks... and it mentions your wife.... so at least the LUGNET software for marital names works correctly! ;-)

    And I am now up to snuff on both.... (Finch and E****... )

    Cheers,
    Gary Istok
  • brickmaticbrickmatic Member Posts: 1,071
    edited May 2011
    Much of Peeron parts inventory is in Brickllink...
    Do you mean the inventory is realized in Bricklink (they share the inventory), duplicated in Bricklink (copied exactly), or replicated in Bricklink (essentially a copy, but built separately)?
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 5,804
    A separately built inventory, which more likely than not is identical to the one at Peeron, is my understanding.

    I think Gary meant 'Much of Peeron parts inventory functionality is in Bricklink...'
  • brickmaticbrickmatic Member Posts: 1,071
    ^ Well my thoughts are if Bricklink is opposed to sharing or providing its database for use at Brickset, maybe Peeron would be more open to it. However, seems the sites are all sort of related and I'm not really understanding all of the relationships going on here. Also, don't know if you want a parts inventory. So, it's likely my thoughts on this are silly.
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 5,804
    edited May 2011
    I do want parts inventories, very much. I have asked Dan about using the Peeron ones but I didn't get a response. I don't know him well enough to keep badgering him about it, but maybe someone here does.

    I'd be willing to license them them in some way so that Dan can invest in Peeron's infrastructure, or take over the Peeron website's functionality completely.

    The relationships are as follows, as far as I know:

    - Lugnet and Peeron share a set database

    - Peeron has its own parts and inventory database

    - BrickLink is entirely separate from every other site and appears to want to remain that way

    - Brickset is independent of all these but attempts to tie them together through links and import/export functionality. Brickset currently imports a minifig database and a minifig-to-set cross-reference, and displays BrickLink minifig images when it doesn't have its own. It also has functionality to display BrickLink inventories but has not yet reached agreement with them to make them available to all. Only a handful of people, including you, can see them.
  • davee123davee123 USAMember Posts: 791
    edited May 2011
    I do want parts inventories, very much. I have asked Dan about using the Peeron ones but I didn't get a response.
    I'm honestly not sure where they stand on licensing out their data-- it's probably been a couple years since I last really talked to them about the whole situation.
    The relationships are as follows, as far as I know:

    - Lugnet and Peeron share a set database
    More specifically, LUGNET's database gets pushed to Peeron, but Peeron has the *ability* to override LUGNET set data. Dunno how often the override is used, though.

    Also, for the record, BrickLink was not *supposed* to use Peeron inventories, but did in some instances. BrickLink itself isn't to blame, but specific BrickLink members are. At the time when BrickLink came out with inventories, they explicitly did not use Peeron data to keep their independence (there's more to that story, but I don't know all the details). Instead, they asked for submissions from BrickLink members, and each inventory submission gave them a credit on BrickLink (discount in BL fees, I think?)

    However, you can take a guess as to what some BrickLink members did. They just manually took Peeron's data and re-entered it into BrickLink. I recall seeing the *exact* same errors in Peeron inventories and BrickLink inventories some years ago, proving the fact. I don't know how widespread the replication was (clearly some were independently inventoried), but it was there. I also heard reports of images having the same issues (IE copying Peeron data to BL), but I don't remember researching those myself-- that was just hearsay from my perspective (and not really surprising).

    DaveE
  • drdavewatforddrdavewatford Hertfordshire, UKAdministrator Posts: 6,002
    edited May 2011
    Also, for the record, BrickLink was not *supposed* to use Peeron inventories, but did in some instances. BrickLink itself isn't to blame, but specific BrickLink members are.
    DaveE
    How ironic, given the curmudgeonly attitude of some of the Bricklink community to Brickset's request to use 'their' inventory data......an objection which looks even more absurd and unreasonable given this latest information.
  • rocaorocao Administrator Posts: 4,274
    She's *my* wife now, as it turns out :)
    ^This, for me, definitely makes the following more clear:
    My actual count is 697,163 for my personal collection, but with my wife's collection added (we're in the process of combining), we're around 1 million.
    I recalled when I first joined Lugnet I looked up Suzanne's collection and it was massive!
  • brickmaticbrickmatic Member Posts: 1,071
    @bluemoose Did you ever get around to asking through the ambassador program how TLG feels about releasing digital copies of their retired catalogs and advertisements for online historical preservation (previously mentioned in another thread)? While you're at it might want to ask how they feel about sharing parts database things, like a listing of element IDs and descriptions. Nothing too problematic. I'm sure they have a list of pictures linked to element IDs, maybe they would feel alright letting Brickset use that as well. Or at least let us use LEGO Digital Designer to capture images of parts for a database. Something like that could be scripted.

    I think TLG likes the existence of Brickset. It furthers their brand at no cost to them and Brickset plays nice. Can't hurt asking if they would mind helping Brickset out with permission to use some of their intellectual property in a limited fashion per their terms. Seek and you shall find, as the saying goes.
  • IstokgIstokg MichiganMember Posts: 1,893
    One AFOL with a large collection of LEGO sets is John Patterson. He is in the process of selling his entire collection (hence my comment about removing his set ownership comments on the LUGNET Set owners page). John had contributed a lot of set inventories to Peeron... but got angry when his replies to Peeron Admin went unanswered (as were Huw's and my own). I even tried to get the ex-wife of the Peeron owner to have him return my Emails... but still no response.

    So John P. stopped contributing to Peeron , and had many of his same LEGO set inventories put into Bricklink. And in the case of any mistakes in his submitted Peeron inventories... they also got into Bricklink. Now granted this is just one scenario of how some errors made it into Bricklink that were previously on Peeron. I'm sure there are others...

    And Drdavewatford... yes there are several ironies in Bricklink... such as those folks who want all Bricklink images either watermarked, or more protected. The absurdity there is that many images have been given by AFOLs to Lugnet and Peeron as well. And the image ownership issue cannot be at all easily settled with the existing image database. The Bricklink Admins know this, and to them image ownership is a non-issue.

    As for the hardliners in Bricklink... some have been banished from posting. So there are now fewer curmudgeons left to protest the interlinking between Bricklink and other sites. As soon as things settle down again over there, the issue will be re-introduced.

  • bluemoosebluemoose Member Posts: 1,716
    @bluemoose Did you ever get around to asking through the ambassador program how TLG feels about releasing digital copies of their retired catalogs and advertisements
    Yes, about 30 seconds after that previous discussion. The CEC team haven't been very active with the Ambassadors recently, so I haven't had a reply. I've poked them again for an answer.
    While you're at it might want to ask how they feel about sharing parts database things, like a listing of element IDs and descriptions.
    I asked them this months ago; the answer was 'no', as they weren't prepared to put the effort into pulling the information together for release to the community.

  • davee123davee123 USAMember Posts: 791
    edited May 2011
    Can't hurt asking if they would mind helping Brickset out with permission to use some of their intellectual property in a limited fashion per their terms.
    I definitely agree-- if someone in the community is trustworthy and has easy access, this would be a great boon to LEGO hobbyists.

    From LEGO's perspective, I believe the only real roadblock might be cost of access. As I understand it, LEGO has a great database of sets and inventories stored electronically through the early-or-mid-1990's. But BEFORE then, records are sketchy, if extant, and would likely be in physical media. And that means having paid LEGO employees overseeing and coordinating the process of pulling up old stuff. But hopefully that's minimal, or someone in the company will be friendly enough to volunteer on their off-time :)
    Now granted this is just one scenario of how some errors made it into Bricklink that were previously on Peeron.
    Possibly-- although I think John's inventories were typically pretty accurate.

    Didn't realize he was selling his collection, though! From the sounds of it, he had a pretty extensive vault of sets! I can only think of one other hobbyist who supposedly had a similarly complete collection.

    DaveE
  • brickmaticbrickmatic Member Posts: 1,071
    @bluemoose Thanks for the effort! Too bad it didn't work out for the database.

    Also, like I mentioned, scripting the extraction of an element database from the element library in a LEGO design program is one way of addressing how to put together an part database easily. LDRAW could be used. They even have licensing information and author information embedded in each part file, which could be checked. I'm afraid, however, most people would opt for a license that precludes the sort of usage envisioned.
  • bluemoosebluemoose Member Posts: 1,716
    @bluemoose Thanks for the effort! Too bad it didn't work out for the database.
    We haven't given up yet ;-)
  • rocaorocao Administrator Posts: 4,274
    @Istokg: is John's sale public? If so, can you give me the details?
  • brickmaticbrickmatic Member Posts: 1,071
    The PAB store could be crawled to extract information for a large chuck of parts. Of course doing so without permission would be bad form and using the crawled information without permission would be infringement. However, the technical means for a DIY extraction are there. I'm really surprised getting together the info is a lot of effort on their part.

    Actually, TLG does not object to scanning and use of their instructions when in good faith according to the Fair Play document. These days instructions include inventory pages. So, scan and process with some detection algorithms and OCR and you can automate the acquisition of both the inventory for the set and some pictures for the database. Just an idea.
  • IstokgIstokg MichiganMember Posts: 1,893
    John Patterson is selling the bulk of his collection in his Bricklink Store....
    http://www.bricklink.com/store.asp?p=legoman77

    Read his splash page for info about what he sells on Ebay as well.
  • IstokgIstokg MichiganMember Posts: 1,893
    edited May 2011
    If you're looking at old LEGO items... John has quite an inventory...
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 5,804
    edited May 2011
    Do you think there's appetite in the community to start a third inventory repository, one which is open and free for anyone to use in any way the see fit?

    I have Dan and Jen on Facebook and have just asked them to read this discussion and perhaps provide input as to how we might move this forward, maybe by licencing their data or 'buying out' Peeron, because it's a great resource and I, and everyone here by the sounds of things, doesn't want to see it die.

    MattDawson
  • davee123davee123 USAMember Posts: 791
    I would guess that people will be happy to use a better system, if it's available-- but it's a lot of work to get there.

    People typically complain that BrickLink is difficult to find the elements they're looking for, and it doesn't have the same friendly cross-referencing tools that Peeron has. And of course everyone complains that Peeron is too slow, and some complain that its data isn't up to date.

    A merging of the two (good data with a good interface) would certainly get used by the community, but it takes a while to get there. Initial usage would probably be pretty low until it became reasonably complete data-wise, or if it were tied into BrickSet, which has high enough usage to get visibility.

    DaveE
  • drdavewatforddrdavewatford Hertfordshire, UKAdministrator Posts: 6,002
    edited May 2011
    I have Dan and Jen on Facebook and have just asked them to read this discussion and perhaps provide input as to how we might move this forward, maybe by licencing their data or 'buying out' Peeron, because it's a great resource and I, and everyone here by the sounds of things, doesn't want to see it die.
    Not me, for one.... the fact I still use Peeron despite the almost glacial speed of the site at times, the well-documented inaccuracies, and the sad reality that it's getting more out of date every day, is testament to my affection for it. Specifically, I love the fact that the combination of set inventories, plus the ability to upload one's set list AND loose parts opens up a multitude of possibilities simply not available anywhere else. Throw into the mix the simple, unfussy user interface and the totally uncommercial feel of the whole thing, and you've got a complete gem of a site.

  • brickmaticbrickmatic Member Posts: 1,071
    edited May 2011
    I think there is an appetite for a complete, accurate, and linked (to sets and other data, like ownership data here in Brickset) inventory. I like Peeron, but it is stagnant. Bricklink is fine, but they don't want to link the data. TLG likely has one too, but for internal use only. So if the existing inventories cannot grow, develop, and move closer to the ideal, then a third repository would be a way to overcome the limitations of the existing resources. If it were free and open, that would be wonderful because it would be the best way to ensure future improvements. I'm all for another database.
  • drdavewatforddrdavewatford Hertfordshire, UKAdministrator Posts: 6,002
    ^ Me too, although given the work involved, if an existing resource (e.g. Peeron) could be used as a foundation, it would give the project a massive kick-start and take us from the realms of fantasy to distinct possibility in one fell swoop.........
  • georgebjonesgeorgebjones Member Posts: 224
    ^^^^^^^Ha! I didn't even make the connection to this discussion, but I just bought a Ton of stuff from John last week. Fantastic prices, huge quantities. Lots of old rare items, well, fewer now. :) FYI, his splash page is out of date, it says his ebay username is Legonut77, but he was forced to change it and is now BillundBricks77.
  • georgebjonesgeorgebjones Member Posts: 224
    @Rocao - Aren't you glad I'm... like... the only one using peeron for set lists? It's amazing what ideas can spring forth in these discussions. Very good stuff.
  • sas0214sas0214 USAMember Posts: 28
    I don't use Peeron, but I do use Bricklink. My major problem is that TLG uses one color name and Bricklink uses a different one. I have just come out of my dark period and am pieceing sets together, so this would help me. And it will also help with my OCD tendency as well.
  • bluemoosebluemoose Member Posts: 1,716
    ^ The Peeron colour chart is your friend :-)
    http://www.peeron.com/inv/colors
    It's not 100% up to date with the newest colours, but it's pretty good.
  • rocaorocao Administrator Posts: 4,274
    edited April 2012
    Here is a sample of a collection list exported from LUGNET, which can be used to import into Brickset. Apologies for it being somewhat lengthy, but it serves the purpose of presenting the format that is in use.
  • HarlquinthHarlquinth Member Posts: 8

    @rocao, Just been looking for a way of importing a list of sets into brickset, but get 'page not found' from that sample - is there an updated link somewhere? (Or is there another way of importing a list of sets?)

  • MattDawsonMattDawson Solihull, UKMember Posts: 972
    Perhaps we should crowd-fund a Brickset acquisition of Peeron? I'd certainly be willing to contribute to it! 

    You could reward people with mentions, site badges and physical merch as rewards...
    sid3windr
«1
Sign In or Register to comment.
Recent discussions Categories Privacy Policy