[Tool] EmpyrionStuff - EPB read/write

Discussion in 'The Hangar Bay' started by Apan Loon, Jun 7, 2018.

  1. Apan Loon

    Apan Loon Lieutenant

    Joined:
    Jun 7, 2018
    Messages:
    80
    Likes Received:
    31
    Absolutely. This program can read many of the templates too. But there are many more older versions of the epb format in there so many files can't be read accurately.

    Edit: And I am pretty sure that templates can be replaced with new models for modding.
     
    #21
  2. Apan Loon

    Apan Loon Lieutenant

    Joined:
    Jun 7, 2018
    Messages:
    80
    Likes Received:
    31
    After a long break, I started to look at this project again. I added reading of the damage states, but it doesn't store or write these yet. Thanks @geostar1024 for this find.
     
    #22
    geostar1024 likes this.
  3. Apan Loon

    Apan Loon Lieutenant

    Joined:
    Jun 7, 2018
    Messages:
    80
    Likes Received:
    31
    Triple-post, but now I investigated more files of older format versions and corrected my read code. Only two blueprints provided by the game crashes my reader now: Prefabs/BAO_AntarisSpacefarm.epb and PrefabsStock/BA_Prefab_Tier5a/BA_Prefab_Tier5a.epb

    Many things are still not read correctly though, but making it process all the bytes of every file I come across is a good start.

    Also, I am pretty sure that what I called "Unknown06" is actually creature spawners, shouldn't be too difficult to figure out more about these.
     
    #23
  4. Apan Loon

    Apan Loon Lieutenant

    Joined:
    Jun 7, 2018
    Messages:
    80
    Likes Received:
    31
    I'll just keep talking with myself, will I? :)

    I figured out how blocks are referenced in the device group list as well as in the signal and logic lists. To make sense of the data, I read the four bytes as an UInt32, then I mask out the following bits:

    uuuu xxxxxxxx yyyyyyyy vvvv zzzzzzzz

    where u and v are unknown, typically bin(1000), i.e. 8. x, y, and z are coordinates into the matrix of blocks.

    I also realised that the device groups have one fewer bytes in the unknown block in file format version 14 and older. Taking this into consideration makes the program able to read all but two of the EPB files provided by the game without crashing. These files are "Prefabs/BAO_AntarisSpacefarm.epb" and "/PrefabsStock/BA_Prefab_Tier5a/BA_Prefab_Tier5a.epb". The first fails while reading "Unknown07" and the latter fails while reading "Custom".

    EDIT: I just saw that the previous post also list those files as the only ones that fail. This probably means that a game update has added a file with version 14 that I never had before. My program failed to read that, but now it can.

    EDIT 2: Oops. Seems as if "PrefabsStock/CV_Prefab_Tier1a/CV_Prefab_Tier1a.epb", "PrefabsStock/HV_Prefab_Tier6a/HV_Prefab_Tier6a.epb" and "PrefabsStock/SV_Prefab_Tier6/SV_Prefab_Tier6.epb" also fails reading the custom bit. It appears as if "Unknown09" only exist in some files and when it doesn't, it messes up the counter.

    "Prefabs/BAO_GhostRiderShipyard.epb", "Prefabs/BAO_Icarus-X2.epb", "Prefabs/BAO_Migotzu-Orbital-Hub.epb", "Prefabs/BAO_OrbitalSpire.epb", "Prefabs/BAO_RadosRepairStation.epb", "Prefabs/BAO_Thermica-Station.epb", "Prefabs/BAW_CrashedCockpit.epb", "Prefabs/BAW_OldBase.epb", "Prefabs/BAW_Opportunity.epb", "Prefabs/BAW_VeryOldBase.epb", "Prefabs/BA_AbandonedFactory.epb", "Prefabs/BA_AbandonedFactory_old.epb" and a whole lot of others also crashes. Mostly when reading spawners and/or logic stuff.

    My mistake was that the UTF8 coding of some display panel text made "grep" assume that my output log was binary and thus it didn't match properly. with "grep -a" I see all the errors... A lot left to do...
     
    #24
    Last edited: Aug 6, 2018
    geostar1024 and Exacute like this.
  5. Apan Loon

    Apan Loon Lieutenant

    Joined:
    Jun 7, 2018
    Messages:
    80
    Likes Received:
    31
    Changed the funky stuff around "unknown02". It now reads the counters instead of just a block of bytes. Thanks to @geostar1024 for leading the way.

    I currently read the following after the tag list:
    Code:
    UInt16 unknown02
    UInt32 nLights
    UInt32 unknownCount01
    UInt32 nDevices
    UInt32 unknownCount02
    UInt32 nBlocks
    UInt32 unknownCount03
    UInt32 unknownCount04
    UInt16 nBlockCounts
    ...
    "unknownCount03" only exists if the version is greater than 14 and "unknownCount04" only exists if the version is greater than 18. Do we know what the counters actually mean? @geostar1024 : How certain are you that "unknown04" is the number of triangles? The value sometimes feel a bit low for that, I think.

    Writing EPBs is also slightly changed - the block count is written correctly and the creation date and time are stored in the new file.
     
    #25
  6. geostar1024

    geostar1024 Rear Admiral

    Joined:
    Jan 24, 2016
    Messages:
    6,291
    Likes Received:
    9,087
    I'm pretty sure about the counters that I have in my code; they match with what is shown in-game when examining a blueprint.
     
    #26
  7. Apan Loon

    Apan Loon Lieutenant

    Joined:
    Jun 7, 2018
    Messages:
    80
    Likes Received:
    31
    Cool, I didn't even know that the "Triangle Count" was in the statistics page. I will rename "unknownCount04" accordingly. I suppose that Empyrion is a lot more low-poly than I thought. :)

    Still, there are some counters(?) that I see values for (not all 0) between the ones you name in your code - I guess that you, like me, still don't know what they mean?
     
    #27
  8. geostar1024

    geostar1024 Rear Admiral

    Joined:
    Jan 24, 2016
    Messages:
    6,291
    Likes Received:
    9,087
    Yeah, I have no idea what the others are; I seem to remember them being either zero or a constant across all the blueprints that I looked at. I think my blueprint writer writes them as zeros anyway, and I don't recall seeing any negative effects when I loaded them in-game, probably because what the game really cares about is the zipped data itself.
     
    #28
  9. Apan Loon

    Apan Loon Lieutenant

    Joined:
    Jun 7, 2018
    Messages:
    80
    Likes Received:
    31
    Ok. Batch-examining all the Prefabs and PrefabsStock files gives me different values for these counters. Still, I don't see a correlation with anything else.

    I am sure that you are right and most of the info outside the zipped part is just for display in the list of blueprints - except the device group definitions, of course. I also save zero in most of these and the game appears to be happy anyway.
     
    #29
  10. geostar1024

    geostar1024 Rear Admiral

    Joined:
    Jan 24, 2016
    Messages:
    6,291
    Likes Received:
    9,087
    It's possible that the triangle count is done differently now, and the values in the blueprint file itself aren't up-to-date anymore, which could explain one of the discrepancies. The block, device, and light counts shouldn't change, though, so that's strange. . . .

    EDIT: oh, you're talking about the unknown counters, so never mind . . .
     
    #30
  11. Exacute

    Exacute Rear Admiral

    Joined:
    Feb 17, 2017
    Messages:
    1,050
    Likes Received:
    760
    There's atleast
    -size
    -triangles
    -lights
    -blocks
    -devices

    Possibly
    -coretype
    -lasttouched (which is a long, atleast in the API)

    These are atleast exposed in the API easily, so I'm guessing it's BP info, as it's also affected by DSL
     
    #31
  12. Exacute

    Exacute Rear Admiral

    Joined:
    Feb 17, 2017
    Messages:
    1,050
    Likes Received:
    760
  13. Apan Loon

    Apan Loon Lieutenant

    Joined:
    Jun 7, 2018
    Messages:
    80
    Likes Received:
    31
    I have never even looked at the API. Maybe I should check that out.

    The size and ”coretype” is defined right at the beginning of the file and the “last touched” is one of the “tags”. I was currently just asking about the seven 32 bit values between the “tags” and the block type counts.

    Edit: Out of those seven, only three are unknown at this time.
     
    #33
    Exacute likes this.
  14. Exacute

    Exacute Rear Admiral

    Joined:
    Feb 17, 2017
    Messages:
    1,050
    Likes Received:
    760
    Yeah, I was just throwing thoughts around :)
    But I'm somewhat sure that the file contains info on the others aswell, based on it being affected by DSL, wether you can access the data or not
     
    #34
  15. Exacute

    Exacute Rear Admiral

    Joined:
    Feb 17, 2017
    Messages:
    1,050
    Likes Received:
    760
    If you want, just make a simple one that have a game_start event, GameAPI.Game_Request(CmdId.Request_GlobalStructure_List, 10, null);
    and a

    case CmdId.Event_GlobalStructure_List:
    GlobalStructureList gi = (GlobalStructureList)data;
    in the game_event section
    You can do it somewhat quick and dirty, although it's not reading the BP files, rather it's reading what is already spawned (ie. you have their 'names', not their file/etc)
     
    #35
  16. Apan Loon

    Apan Loon Lieutenant

    Joined:
    Jun 7, 2018
    Messages:
    80
    Likes Received:
    31
    Yes, the ones you listed have been confirmed. I have one weird example where the “Triangle Count” is zero in the file though. This is a base with only a core block. But other than that, this counter appears to correlate to the triangle count listed in game.
     
    #36
    Exacute likes this.
  17. Exacute

    Exacute Rear Admiral

    Joined:
    Feb 17, 2017
    Messages:
    1,050
    Likes Received:
    760
    Hmm.. that almost gotta be a bug, but assumingly, they are just not counting the core itself, as it is "obligatory"?
     
    #37
  18. geostar1024

    geostar1024 Rear Admiral

    Joined:
    Jan 24, 2016
    Messages:
    6,291
    Likes Received:
    9,087
    Do devices even contribute to the triangle count? I had thought that they don't, in which case this would be the expected behavior.
     
    #38
  19. Apan Loon

    Apan Loon Lieutenant

    Joined:
    Jun 7, 2018
    Messages:
    80
    Likes Received:
    31
    Actually the core isn’t required. Lots of EPBs in the game have no core.

    It is possible that this example is not 100% correct though. It was a long time since I made that blueprint and I might have done stuff(tm) to it.
     
    #39
  20. Apan Loon

    Apan Loon Lieutenant

    Joined:
    Jun 7, 2018
    Messages:
    80
    Likes Received:
    31
    @geostar1024 that would explain both my example and my feeling of the number being too low.

    Edit: It would reduce the meaning of the statistic though - what is the point of having a “render cost estimate” that doesn’t include all triangles?

    Oh well, the reasoning for this is interesting but not very important.
     
    #40

Share This Page