Not signed in (Sign In)

Vanilla 1.1.10 is a product of Lussumo. More Information: Documentation, Community Support.

    •  
      CommentAuthoressen
    • CommentTimeMar 17th 2010
     
    Continuing the discussion started in the following thread: http://forum.psupedia.info/viewtopic.php?f=22&t=855

    Tools repository: http://github.com/essen/gasetools

    Current support include the decryption, decompression and extraction of most files except the textures because they're handled a bit differently than the rest. Current goals include extracting the textures and allowing the creation of archives usable by the offline game executable, but also giving each file an accurate description of what it contains.
    •  
      CommentAuthornyu-zero
    • CommentTimeMar 17th 2010
     
    This thing supports PSPo too?
    •  
      CommentAuthoressen
    • CommentTimeMar 17th 2010
     
    Currently initial work has been done to extract nbl files from PSP1 and PSP2, and those files can be extracted just like AOTI files. They use the same encryption and format, meaning we can't extract the textures in those yet.

    Note though that the PSP extractor currently only extracts nbl files, there might be other formats too.

    Mewn did a preliminary study of those files and could give you more info. He found files with NPC names and stuff.
    •  
      CommentAuthornyu-zero
    • CommentTimeMar 17th 2010
     
    Well I was worried more about the huge zip like file. May be of interest later if you know what i mean ;)
    •  
      CommentAuthoressen
    • CommentTimeMar 17th 2010
     
    What huge zip like file? I have only checked on PSP2 and assumed PSP1 was the same. PSP2 has an fpb file which is basically the game executable + data.
    • CommentAuthorMewn
    • CommentTimeMar 17th 2010
     
    The info I got on PSP2 (no PSP1 stuff, wasn't interested in that) was pretty basic, but I did find stuff like NPC names, item and enemy names and the like. Data is on my laptop though and I cba to get it from there right now, I'll grab it tomorrow midday-ish.
    •  
      CommentAuthornyu-zero
    • CommentTimeMar 18th 2010
     
    [quote][cite] essen:[/cite]What huge zip like file? I have only checked on PSP2 and assumed PSP1 was the same. PSP2 has an fpb file which is basically the game executable + data.[/quote]
    Back when PSPo1 was released i immediately checked the iso file for some interesting things but only found a huge compressed file (not the main binary) Also i dont remember what extension it had. Main focus of interest is still at PSPo2 for me regarding this ;)
    •  
      CommentAuthoressen
    • CommentTimeMar 18th 2010
     
    Shouldn't be too hard even if it's different. To extract from PSP2 I just know the padding size and check at every possible padded positions in the file for the nbl file identifier, and extract everything until the next as one nbl file. It works well apparently, though I think some files were crashing the program. Either way most files are added to PSUJP currently so there's no real need to go through the PSP2 files to get the various items.
    • CommentAuthorMewn
    • CommentTimeMar 18th 2010 edited
     
    Edit: This is for PSP2. Didn't bother looking at PSP1.

    Had a lot less data collected than I thought I did. Ok, so if you extract the fpb file with essen's tool it'll give you a numbered list of files. The following numbers were of interest to me:

    3.nbl - CharaSaveWorkFlag.ubr - appears to contain flags of various sorts (rare block stamps, story mode clear, FRENDSHIP)
    4.nbl - Contains what appear to be various animation and reference files. Not entirely certain.
    5/6.nbl - Img files for eyes, eyebrows and other facial features.
    7.nbl - Files for mode select, options, install etc.
    8.nbl - Files for matching counter. No discernable text beyond filenames and stuff like close_loop.
    9.nbl - Full of .rel filelists.
    11.nbl - param .unr files for various item types, as well as shop lists and PA caps. Unintelligable.
    12.nbl - EnemyLevelBaseParam.unr
    13.nbl - EnemyBoostParam.unr and EnemyLevelTable.unr
    16.nbl - Apctext_J.bin - contains partner character names and full names
    21.nbl - Contains .ubr files for partner machines.
    22.nbl - Appears to contain NPC chatter and quotes.
    275.nbl - visual phone text (contains a lot of stuff, enemy descriptions, title names and descriptions etc.)
    397.nbl - Contains type customization counter text.
    399.nbl - Contains character import text.
    400.nbl - Contains the Japanese credits file
    2522.nbl - item/enemy/general strings (also contains a lot of stuff)

    There's probably a fair bit more but I got bored so eh. Text-based ones can be read in a hex editor (with the correct text encoding) but I'm sure you guys would know that.
    •  
      CommentAuthoressen
    • CommentTimeMar 18th 2010 edited
     
    I figured someone wanted to do art so I documented this based on the source I already had (included at the end): http://psumods.co.uk/wiki/files:launcher
    •  
      CommentAuthornyu-zero
    • CommentTimeMar 18th 2010
     
    Guess what the server sends to the client! Yeah right! .nbl files! Same structure and same gibberish to me like everything else regarding this.

    Here are two of them you get when you join Holtes East on the english PC version (note the PS2 version is a bit different)
    http://pastebin.com/n3MUQ5jt
    •  
      CommentAuthorDarK-SuN
    • CommentTimeMar 18th 2010 edited
     
    Maybe it's sending quest files, much like the ones available offline, but with the online data (excluding graphical data since that's in the disk already).
    It's more or less accepted by some people that everything is handled like a "QUEST", even regular areas like the Guardians HQ and so on, but instead of being "battle mode quests", they're "exploration mode quests" for walking around and so on.
    This is more or less supported by the fact that the MAG mission, at the end, has a mission counter that leads to Program DF, so it's a counter inside a mission already.
    Basically, everything is a mission, or, more like, "state" configuration for the area, so even missions are just a series of accessible areas with a specific state and rates, but we access them like missions (battle mode state).
    •  
      CommentAuthoressen
    • CommentTimeMar 18th 2010
     
    What you say exactly. It's not "more or less accepted", I can confirm that it's exactly that. Even cutscenes work like this.

    Anyway I've finished describing everything I know about the nbl format so far: http://psumods.co.uk/wiki/files:nbl
    •  
      CommentAuthoressen
    • CommentTimeMar 18th 2010
     
    Another cool thing is that while most nbl files are encrypted and compressed, PSU can also simply load unencrypted and uncompressed files, so that theoretically removes a lot of work in building a packer. I have some code for that already but using it lead to my discovery of the TMLL chunk containing textures, and I haven't come back to it since then.
    •  
      CommentAuthornyu-zero
    • CommentTimeMar 18th 2010
     
    DarK-SuN:Maybe it's sending quest files, much like the ones available offline, but with the online data (excluding graphical data since that's in the disk already).
    It's more or less accepted by some people that everything is handled like a "QUEST", even regular areas like the Guardians HQ and so on, but instead of being "battle mode quests", they're "exploration mode quests" for walking around and so on.
    This is more or less supported by the fact that the MAG mission, at the end, has a mission counter that leads to Program DF, so it's a counter inside a mission already.
    Basically, everything is a mission, or, more like, "state" configuration for the area, so even missions are just a series of accessible areas with a specific state and rates, but we access them like missions (battle mode state).

    Yeah I had the same feeling regarding this so it is more or less confirmed. Hehe, how about a lobby full of monsters? :D That is definitely something I WILL try :3
    •  
      CommentAuthornyu-zero
    • CommentTimeMar 18th 2010
     
    essen:Another cool thing is that while most nbl files are encrypted and compressed, PSU can also simply load unencrypted and uncompressed files, so that theoretically removes a lot of work in building a packer. I have some code for that already but using it lead to my discovery of the TMLL chunk containing textures, and I haven't come back to it since then.

    It would be interesting if it will be possible to send more than just quest data to the client. I am still far from that though :s I just finished correct login handling.
    •  
      CommentAuthoressen
    • CommentTimeMar 18th 2010
     
    I doubt you can send more than the client expects. The patch server handles everything else.
    •  
      CommentAuthoressen
    • CommentTimeMar 19th 2010
     
    I've made stats from the files, all files that use a TMLL chunk (1831) use compression in that chunk. So the TMLL decompression algorithm is needed before going further. I guess I'll go back to the boring part of the research in a few days. Will be away most of the week-end anyway.
    •  
      CommentAuthorDarK-SuN
    • CommentTimeMar 19th 2010
     
    Cool, thanks for confirming that, Essen. :)
    Had that feeling for a while, specially because in the PC version, on ocasion, you could see what's under the "Now Loading" screen and everything was, geometry wise, already loaded, meaning what was going on was just the NBL with the area configuration/mission configuration being loaded to put down the entities and counter information.

    Nyu-Zero, if I'm not mistaken, there's a Guardians Cash mission in the JP server that already has monsters in the Guardians Colony HQ, so that's very much possible (please correct me if I'm wrong).

    The non-SEED infected Linear Line files are all in PSU (because of PSUv1 offline story mode, as well as for midpoint lobbies that use it), meaning that we could even use them as another "battle mission" by placing monsters/NPCs in them, or simply to expand the normal exploration areas of the Colony. :)
    •  
      CommentAuthoressen
    • CommentTimeMar 19th 2010
     
    The last story mission of Episode 2 has battles taking place in the Guardian's Colony and Parum. Also the Icebreak event from last summer on JP had a bonus mission taking place in Vio Tonga (the snowy lobby on Moatoob). It's clearly possible without any problem.

    However the lobbies aren't really designed for missions (small and short route from point A to B), so their use is limited.
    •  
      CommentAuthorQwerty
    • CommentTimeMar 19th 2010
     
    Some interesting things could be done using lobbies as a sort of final area of a mission without an actual boss, just a bunch of monsters. Obviously, most of the lobbies are either too small or virtually indistinguishable from rooms of mission maps, but I could see some fun stuff going down on Colony floor 1 or one the observation areas.
    •  
      CommentAuthornyu-zero
    • CommentTimeMar 19th 2010
     
    Qwerty:Some interesting things could be done using lobbies as a sort of final area of a mission without an actual boss, just a bunch of monsters. Obviously, most of the lobbies are either too small or virtually indistinguishable from rooms of mission maps, but I could see some fun stuff going down on Colony floor 1 or one the observation areas.

    I plan to try to spawn a lot of rappies down there just to test things out :D Or maybe a dragon :O that would be...funny :3
    •  
      CommentAuthorDarK-SuN
    • CommentTimeMar 19th 2010
     
    Yeah, that'd work quite well, Qwerty, specially for "community event missions" type of thing. :)
  1.  
    has anyone managed to extract the files from "file 2" from that packet log that nyu-zero posted? (http://pastebin.com/n3MUQ5jt )
    I remove the first line of the consecutive packets, then pasted it into winhex. The nbl tool worked fine of the first file, but the second file will only list 3 files but not extract them.
    Server -> Client, Command 0x0B030000, Length 00004010b, 16400 bytes
    00000000 | 10 40 00 00 0B 03 00 00 38 10 01 00 00 40 00 00 | .@......8....@.. <-------- I remove this line (assuming its the complete packet header)

    either i`m not removing all the packet header or log on pastebin for file 2 is missing a bit IDK.
    I`m going offline for a bit soon and i could have done with a couple of online quest files to analyze when i`m bored while i`m offline.
    •  
      CommentAuthoressen
    • CommentTimeMar 19th 2010
     
    I don't see anything problematic in the file. Maybe a bug in the extractor, I'll have to check it out.
    •  
      CommentAuthornyu-zero
    • CommentTimeMar 19th 2010
     
    scriptkiddie:has anyone managed to extract the files from "file 2" from that packet log that nyu-zero posted? (http://pastebin.com/n3MUQ5jt )
    I remove the first line of the consecutive packets, then pasted it into winhex. The nbl tool worked fine of the first file, but the second file will only list 3 files but not extract them.
    Server -> Client, Command 0x0B030000, Length 00004010b, 16400 bytes
    00000000 | 10 40 00 00 0B 03 00 00 38 10 01 00 00 40 00 00 | .@......8....@.. <-------- I remove this line (assuming its the complete packet header)

    either i`m not removing all the packet header or log on pastebin for file 2 is missing a bit IDK.
    I`m going offline for a bit soon and i could have done with a couple of online quest files to analyze when i`m bored while i`m offline.

    You should probably remove everything before the NMLL part (i assume that .nbl files start with these 4 ASCII letters). The first file would be starting at 0x34 and the second file would be starting at 0x44.
  2.  
    i did. the problem is either how i reconstructed the 2nd nbl file or a possible bug in essens extractor
    •  
      CommentAuthoressen
    • CommentTimeMar 19th 2010
     
    I'll try it sometimes next week.
    •  
      CommentAuthorQwerty
    • CommentTimeMar 19th 2010
     
    Qwerty:Some interesting things could be done using lobbies as a sort of final area of a mission without an actual boss, just a bunch of monsters. Obviously, most of the lobbies are either too small or virtually indistinguishable from rooms of mission maps, but I could see some fun stuff going down on Colony floor 1 or one the observation areas.
    Oh yeah also Falz Memoria. What a waste of a lobby. Also, could do some fun things using boss arenas for things other than only bosses.
    •  
      CommentAuthorRaujinn
    • CommentTimeMar 20th 2010
     
    Wonder how many monsters you can spawn before psu crashes.. Go all dynasty warriors n shit
    •  
      CommentAuthorRaujinn
    • CommentTimeMar 20th 2010
     
    Not the most interesting of ideas I'm sure, I just want to see how much the client can take >:(
    •  
      CommentAuthoressen
    • CommentTimeMar 21st 2010
     
    On PS2, not much. On PC there's probably more room if there's no hard limits.

    Just thought that it would be good to extract all the script.bin I can find and make some sort of stats out of the files to figure out more about it. For example get a list of the strings there's in it and similar things.
    •  
      CommentAuthornyu-zero
    • CommentTimeMar 22nd 2010
     
    To those who have not yet seen it: http://dl.dropbox.com/u/3656009/nbldump.zip some of the nbl files i could dump out of the network stream. There was no way to identify them correctly ;_;
  3.  
    hmm looked at the "04F9C9CC" file. the files inside it are pretty much the same as mission files. only difference being there no "zone" files

    theres "enemy00_0102_r0.xnr" style files which are in every mission .nbl, due to the size and makeup of these, they kinda remind me of files what tell the game to load stuff "enemys/objects/ect"

    theres also "set_r1.rel" the .rel is commen in pso, its allways a file what say where stuff go like "use this lighting here" or "show this object here"

    it pretty much allways only affect stactic objects though. like lighting or even the wall of the area your in, i dont really think ive ever seen a .rel affect placements of enemys.

    theres also the "script" and "text" files same as the missions.
    this text.dat seems to hold all the choices of going to different worlds/choosing different quests/adding and removing different npc characters.

    dont really kno much about the script file, so cant really say too much on that right now.

    most of what i said might be wrong ect, but thats what the files look like to me, and tbh its makes quite abit of sence too
    •  
      CommentAuthorjericho2442
    • CommentTimeMar 22nd 2010 edited
     
    if we look at how a pso area is made up its pretty much the same.

    first you have the .rel which tell it which map to load and also how to build all the parts of the map together. think of them as different rooms and ID's and they all need to be told where to go.

    different REL files are telling the rooms where to load and also the lighting ect ect. look at this pic to understand more

    Photobucket

    this is both for the same area but using different .REL files. the .REL's are telling the game where to put each room, thus its making a new route up.
    _______________________________________________________

    then theres the files which load the enemys/the object you can collide with ect.
    on pso enemys are loaded into area's by using files with names like "map_cave01_00_01e" thats the name of the map and area. also what set of rel it will be loaded up by and the "e" at the end stands for "enemy"

    then theres the object files. which loads up particle effects and light sources ect and also object placements, named again the same as the above but insted of "e" they use

    "ad" for particle effects and light sources

    and "o" which load the the objects like teleporters ect

    finally theres the file what is used to create the events in area's

    these are again the same name, but this time they are a .evt insted of .dat

    these hold all the events in the area and w/e esle you need.

    these are the main files in a pso area. and when you look at the way they work and are named, they seem quite the same as psu
    ____________________________________________________________
    most likely the "set_r1.rel" and the "zone" files in mission are the same kind of thing as the .rel's in pso

    the "enemy00_0102_r0.xnr" might be the same as the "map_cave01_00_01e" style files in pso, for loading up how many enemys and which ones

    and then the script and text files for telling what to do, where to fight, when waves appear, and what everyone says

    again i wont say im right, and i may have some facts wrong, but i hope it can help you out at least a tiny bit, since psu seems very close to pso in how its built

    EDIT: srry for all the edits but had to change stuff abit so i got over what i really meant, (im not the greatest writer at all :/)
    •  
      CommentAuthornyu-zero
    • CommentTimeMar 22nd 2010
     
    jericho2442:if we look at how a area is made up its pretty much the same.
    *snip*


    After a LOT of trial and error i now know that nbl files are not neccessasy to make the game load a map (yeah i got this far but im stuck at the loading screen XP at least i can now identify the area...gotta make some manual save function :) for map dumping :D )
    •  
      CommentAuthorjericho2442
    • CommentTimeMar 22nd 2010 edited
     
    nyu-zero:
    jericho2442:if we look at how a area is made up its pretty much the same.
    *snip*


    After a LOT of trial and error i now know that nbl files are not neccessasy to make the game load a map (yeah i got this far but im stuck at the loading screen XP at least i can now identify the area...gotta make some manual save function :) for map dumping :D )


    ya like i said it seems like "some" of the same data as a mission file is being sent. but the main bulk of all the data is in the psu files in your HDD

    also i thought maybe knowing a lil more about how it all works on pso and how close it looks might help you out a bit :)
    •  
      CommentAuthornyu-zero
    • CommentTimeMar 22nd 2010
     
    Good good :3 Also is there a list of map files? I have found a MAP ID field in one packet which tells the client to load a specific map. Loads of crashes can happen if its a wrong ID :/
  4.  
    unfortunatly ive only located every mission file so far, but each of them also seem to contain map information, since psu seems to use missions like pso used area's.

    tell me what mission your trying to load up and maybe you can find the map ID within the file for that mission?
    •  
      CommentAuthornyu-zero
    • CommentTimeMar 22nd 2010
     
    jericho2442:unfortunatly ive only located every mission file so far, but each of them also seem to contain map information, since psu seems to use missions like pso used area's.

    tell me what mission your trying to load up and maybe you can find the map ID within the file for that mission?

    To be honest i can only give you the map ID :( Its 0x2328 or 9000

    I didnt even send mission data as they seem totally different :/
    •  
      CommentAuthoressen
    • CommentTimeMar 22nd 2010
     
    Some maps are identified but their IDs are probably in a different file.
  5.  
    essen:Some maps are identified but their IDs are probably in a different file.


    ya im just not tech savvy enough to find them atm. i kno as soon as essen can extract textures i will be able to tell what every file with textures is and identify as much as possable.
    •  
      CommentAuthornyu-zero
    • CommentTimeMar 22nd 2010
     
    Okay, appearantly everything regarding IDs and map names is in the first .nbl you get from the server (the smaller one). It differs from planet to planet.
  6.  
    nvm
    • CommentAuthorF Gattaca
    • CommentTimeMar 25th 2010
     
    nyu-zero, how are you going about getting the Network Mode .nbl files?
    •  
      CommentAuthornyu-zero
    • CommentTimeMar 25th 2010
     
    F Gattaca:nyu-zero, how are you going about getting the Network Mode .nbl files?

    look in the schtserv forums for what i did :D And by now i must have TONS of data ;)
  7.  
    does anybody know which file has all the item names in it?
    •  
      CommentAuthornyu-zero
    • CommentTimeMar 26th 2010
     
    scriptkiddie:does anybody know which file has all the item names in it?

    *drumroll* ITS NOT NEEDED!!!!
    http://schtserv.com/forum/viewtopic.php?p=594989#p594989
    This post contains item data from my character sent by the server. If you carefully look you see that the psu server merely does a "buffer copy" to display item names to speed things up. There is no ID associated to the item data. At least i think it is like that. I need to get my server to work first correctly :P
  8.  
    i meant for offline mode really. item00ValueData.xnr has all the item codes and offline shop prices and itemObjectDrop.xnr and itemEnemyDrop.xnr for example are just full of item codes. I just wanted the file with the item names so i could make a tool to name everything at the click of a button so i know exactly what im looking at.
    • CommentAuthorLee
    • CommentTimeMar 26th 2010
     
    can I ask a favour, I need to download psu, but in the meantime, if nobody minds me and an old friend want to take a shot at maybe extending qedit's functionality to psu, so if anyone can upload some examples of the map data, some of the more simple quest data (preferably without npc interaction for now) and possibly some item or eemy models if you have id'd anything that would be perfect :P