Hey there, I've got an idea for a script that would create progressively "growing" versions of blueprint files. I'd like to make this script available to the community so they can see what bases look like at different stages, but it requires a lot of work just to digest the information that is in the EPB. a couple of people have created scripts that do similar things, but the quality of their code means that it really hard to expand on their work, which in many cases is buggy and incomplete. Is there any way that you guys can publish some of the system that serializes structures to EPB format? I don't even need the enum that describes which numbers correspond to which block type, all I really need are: * number of bytes in header (bonus if you can describe what the header actually means) * basic format of the block descriptions ** which bytes are block type ** which bytes are block position ** which bytes are block orientation ** (ideally) which bytes are texture/symbol maps
This is what I've got so far (haven't had time to work on it in a while): https://github.com/geostar1024/epbtools Basically, my code can parse the blueprint file and put all the block data (minus symbols, which are weird and I haven't figured them out yet) into an array. In principle, it can also write that block data back to the uncompressed format, but something isn't quite right yet (it doesn't match the original). The reason you can't edit blueprint files directly is because the actual block data is run through PKZIP before being appended to the file. EDIT: this requires python3; all the packages it requires should already be present in a python3 installation. My code isn't as neat as I'd like it, but there are comments at least! EDIT2: url updated
Byte 101 should be grpnameLength, 102+length, should be the name. Another is from 126, 127+length, which also seems to contain grpname That's all I had that you didn't.
I'm pretty sure I'm extracting the group names correctly; could you run it on one of your blueprints and see if it's working properly? The thing that I know I'm not handling right now are renamed devices; pretty sure my code will fail on such a blueprint . . .
You're welcome! If you learn anything about the rest of the stuff in the header, please post it here (or message me) so that I can update my code; alternately, you should be able to fork it and then issue a pull request against my repository. My overall goal is to first create 4 functions (read/write overall epb format, and read/write zipped block data) for working with epb files, and then implement some simple operations for working with blocks like copy, paste, and mass color/texture change, and then link all of this to a room generator to make procedurally-generated POIs (maybe ships too). EDIT: The embarrassing thing is how long it took me to figure out that the block data format was PKZIP; I spent hours trying to tease apart the raw hex, and then hours more looking into various compressed formats, before I happened to hit upon it.
Yeah, umm as a nitpicky thing. PKZIP is software that was written byt the PKWARE software company https://en.wikipedia.org/wiki/PKZIP it used the ZIP file format to store data https://en.wikipedia.org/wiki/Zip_(file_format) which supports a bunch of encodings, in this case, I'm pretty sure (but not at all certain) the block data is compressed using the DEFLATE algorithm https://en.wikipedia.org/wiki/DEFLATE Awesome work though! edit: I have believed a lie for a very long time, PK and PKWARE are different companies
You are completely right on all accounts (and I should update that in my code). Did you get it to run, by the way?
I'm pleased to announce that I have implemented preliminary support for both reading and writing blueprint files! The major limitation at present is that neither groups nor signal logic is extracted from the blueprint file, but everything else is. The resulting blueprint object can then be written back to a file that Empyrion can read. If you're going to modify blueprints with this, I'd definitely recommend reblueprinting the modified structure in-game after spawning it in before doing any more work on it. As the readme describes, I've implemented a couple of functions to do replacing of blocks. It's pretty rudimentary at present, but it does let you do a few things that aren't possible with the in-game console commands, like change all of one block shape into another. https://github.com/geostar1024/epbtools
I've done some work on this for the past few days and figured out a few more things like groups and signal logic. Here are my results so far: EDIT (2017-8-16) updated to include version 18 Code: .epb file format All values are little endian unless specified otherwise. header: magic 4 bytes hex: 45 52 94 78 version 4 bytes known versions: 15, 17, 18 ship type 1 byte width 4 bytes heigh 4 bytes depth 4 bytes (unknown) 114 bytes timestamp 8 bytes (unknown) 9 bytes build 2 bytes (unknown) 3 bytes creator id steamID[1] creator name steamID owner id steamID owner name steamID (unknown) 11 bytes num lights 4 bytes (unknown) 4 bytes num devices 4 bytes (unknown) 4 bytes num blocks 4 bytes (unknown) 4 bytes ! only in version 18 ! num triangles 4 bytes num block types 2 bytes for each block type: block id 1 byte (unknown) 1 byte count 4 bytes (unknown) 1 byte num groups 2 bytes for each group: group name string[2] (unknown) 1 byte (unknown) 1 byte ! not present in version 15 ! num devices 2 bytes for each device: position 4 bytes packed coordinates[3] name string (unknown) 2 bytes The rest of the file is a zip archive, but the first two bytes are missing, they are: 0x50 0x4B ("PK") The zip archive contains a single file with the grid data. grid: The grid file starts with several bitfields. Each bitfield consists of: length 4 bytes usually width*height*depth/8, rounded up bitfield length bytes Then there is an entry for each "1" in the bitfield. The size of the entries varies between bitfields. blocks bitfield, entries: block id 1 byte rotation 1 byte subtype 2 bytes damage bitfield, entries: damage 2 bytes (unknown) 2 bytes color bitfield, entries: color 4 bytes texture bitfield, entries: 8 bytes symbol bitfield, entries: 4 bytes symbol rotation bitfield, entries: 4 bytes Next are several key-value-maps. The format of a map is: (unknown) 1 byte seems to always be 1 num entries 2 bytes for each entry: type 1 byte name string value, depends on type: 0: int, 4 bytes 1: string 2: 1 byte 3: float, 4 bytes 5: color, 4 bytes The block data map contains additional data for some blocks, e.g. lcd panel text and color. num block data 2 bytes for each block data: position 4 bytes packed coordinates map, common entries are: "Text" lcd panel text "ColB", "ColF" lcd panel background/foreground "FS" lcd panel font size "SizeFr", "SizeS1p", "SizeS1n", "SizeS2p", "SizeS2n" sensor area size (unknown) 2 bytes The controls map contains a signal name for blocks that emit signals. num controls 2 bytes for each control: ! in version 15 ! position 4 bytes packed coordinates name string ! in version 17 and 18 ! map, entries are: "Pos" block position, packed coordinates "Name" "State" The signals map contains a list of targets for each signal. num signals 2 bytes for each signal: name string num targets 2 bytes for each target: map, entries are: "Pos" block position, packed coordinates "Func" "Inv" "Beh" The circuits map contains logic circuits like and/or. num circuits 2 byte for each circuit: map, entries can be: "OpName" "OutSig" "InSig0" "InSig1" "InSig2" "InSig3" "FPar" [1] steamID: (unknown) 5 bytes length 4 bytes big endian id length bytes [2] string: The length of a string is encoded with variable length. The low 7 bits of each byte are part of the length, the high bit indicates wether another byte follows. Examples: 20 (0x14) will be encoded as 0x14 127 (0x7F) will be encoded as 0x7F 128 (0x80) will be encoded as 0x80 0x01 129 (0x81) will be encoded as 0x81 0x01 32768 (0x8000) will be encoded as 0x80 0x80 0x02 The length is followed by the string's content. [3] packed coordinates: Packed coordinates are 4 bytes (32 bits). Read as a little endian int, the layout is: | 1 | x, 11 bits | y, 8 bits | 1 | z, 11 bits | (there are two unused bits that seem to always be 1) To unpack the coordinates: x = (packed>>20)&0x7FF y = (packed>>12)&0xFF z = packed&0x7FF Oh and thanks geostar1024 for your work, it was a great starting point
Most excellent! As soon as I get the chance (possibly this weekend), I'll integrate your findings. Thanks especially for figuring out the signal logic and that the groups data contained packed coordinates (it would explain why I had so much trouble with the groups). Also/alternatively, if you feel like writing code, feel free to issue a pull request against the github repo.
Does this mean you wouldn't be able to write a blueprint that had two blocks overlapping? (Or am I misunderstanding the meaning of bitfield? Because it sounds like they are using a 3-d bitmap of which locations do or don't have a block.)
You're understanding it correctly. No overlapping blocks (the game probably can't handle that anyway) You could, however, add a block to multiple groups… someone needs to experiment with things like that By the way, I just downloaded the newest experimental build and there is a new version of the format (18) and all the stock blueprints seem to be converted…
As far as the bitfield is concerned, for multi-block devices it'd just have a bunch of zeros for locations within the multi-block device. Actually, if you modify the block data to produce overlapping devices, the game will still let you spawn in that blueprint (though it tends to give me an internal error when doing so); I've turned a rectangular prism of steel into a bunch of overlapping T2 large generators, for instance. However, it doesn't seem to persist when reloading a save (which makes sense), so there's no practical way to have overlapping blocks in-game.
Ah yeah I guess if you can set the location of bitfields you could have those overlap. I wasn't looking to overlap devices as much as blocks. For example placing two (or four) of the thin walls in different orientations to make a hallway, overlapping blocks to make a transition, or placing a light in the same space as a block. Stuff like that. (Although I suppose windows and lights are devices, so I'd also want to overlap them LOL.) Oh well, it seems to be on many wishlists so maybe they'll make a legit way to do it.
Right, I forgot about multi-block devices. I imagine that would be very difficult to implement and require huge changes to the engine… then again, Medieval Engineers and some Minecraft mods have managed to do this already…
