Announcing amethyst-dynamic-prefab for initial discussion and feedback

(David LeGare) #1

So I made a thing:

This is intended to be the prefab format that users will be able to create and modify through the editor. The existing prefab implementation isn’t ideal for using in the editor for various reasons, so this implementation was designed to unblock some of the work needed to have a prefab editor. On the other hand, this prefab format is pretty impractical to write by hand, so it won’t be ready for wider adoption until we have the prefab editor built.

I initially posted this in Discord and got some replies which I will respond to here:


@Xaeroxe said:

YAY! That pleases me greatly, that was the entire reason I made serde_dyn , so we could do this at some point. Though I’m curious why you feel type-uuid should be a separate crate

I did a couple of things differently than your initial implementation:

  • I had the TypeUuid::UUID constant be a uuid::Bytes rather than a u128, because I immediately ran into endianness issues when trying to compare the string version of the UUID to the u128 version.
  • Rather than deserializing the data into a Box<Any>, I wanted to deserialize it into a trait object that directly implements the methods I need. I don’t know that this approach is actually better than what serde_dyn provides, it just made more sense to me at the time and I was in a rapid-prototyping phase :sweat_smile:

It also occurs to me that type-uuid could have use independent of serde_dyn, so it could be practical to publish it as a separate crate in order to make it the Rust standard for creating stable type IDs.


@kabergstrom said:

The only remaining feature to support my vision of asset pipeline for prefabs will be support for deserialisation of Handle<T>, where the handle is represented using an AssetUUID in serialised form.

This should be pretty doable. I’m still using PrefabData under the hood, which includes a notion of loading sub-assets. So you could create an AssetHandle type and implement PrefabData for it, and it would automatically load the dependent asset.

Oh, and instantiation of Prefabs too.

I think you’re referring to sub-assets here? Nested prefabs? Should already work (or require little extra work to implement) since the existing prefab system supports sub-assets :crossed_fingers:


@jojolepro said:

Did you add prefab editing to the editor too?

Nope! There’s still a couple of major blockers in the way (type schemas and the editor core), but this is the first step toward the prefab editor.


@Xaeroxe said:

We should consider serializing as RON rather than JSON too imo
One argument I can see in favor of JSON is it has bindings for other languages already built

I used JSON here simply because the implementation was more mature. I ran into some issues using ron::Value, and noted that support for untyped desererialization wasn’t fully supported. serde_json is much more mature at this point, so it was more expedient to use it.


@Xaeroxe said:

Also we should restructure the amethyst dependency to not depend on Amethyst directly. That way we can integrate it into the amethyst crate.

I don’t think this needs to be directly in amethyst. amethyst only needs to depend directly on type-uuid such that all component/prefab data types have a UUID. Otherwise this can exist as one of the child crates like amethyst-assets.

1 Like

(Jacob Kiesel) #2

Using uuid::Bytes is probably better and I would be happy to move the serde_dyn crate to use your type_uuid crate.

I was imagining an implementation that would deserialize the UUID and components, then use a similar mapping to look up a function with a uuid that takes an Entity, and a Box<Any> and attaches the type erased component to the entity by downcasting the Box<Any> before attaching it. Such a collection of functions would be implemented with a generic function, like TUSM::register. Basically apply the same trick but for a different task. This approach works too, it just wasn’t what I originally had in mind.

I used JSON here simply because the implementation was more mature. I ran into some issues using ron::Value , and noted that support for untyped desererialization wasn’t fully supported. serde_json is much more mature at this point, so it was more expedient to use it.

Interesting, can you file an issue with the ron repo?

I don’t think this needs to be directly in amethyst. amethyst only needs to depend directly on type-uuid such that all component/prefab data types have a UUID. Otherwise this can exist as one of the child crates like amethyst-assets.

If you look at amethyst_assets it doesn’t depend on amethyst. amethyst depends on it, and we can’t export this through the main amethyst crate until we remove that dependency. As far as I can tell you can just depend on the amethyst sub crates rather than amethyst itself.

0 Likes

(Jacob Kiesel) #3

Also as an addendum: serde_dyn is format independent and can be used with any format, fixed size or not.

0 Likes

(Jacob Kiesel) #4

It’s just occurred to me you might not get my responses until I @randomPoison you.

0 Likes

(David LeGare) #5

Thinking about it more, the advantage of the the Box<Any> approach is that you can use the deserialized data for multiple purposes, since you can still downcast it to its concrete type. So we could use e.g. serde_dyn to deserialize data coming in from networking and prefab data without needing to register the types multiple times. The problem with that, though, is that it would require coordination between the two to use a centralized TUSM instance. That may make sense to do by adding the TUSM as a resource, but it may not end up being beneficial. That’s something I’d want to revisit once we have multiple subsystems using UUID-based deserialization.

I can try to track down the specific issues I was running into, sure :slight_smile: I’ll need to do a little bit of work to make the loading logic agnostic to the format (currently it’s hard-coded to use serde_json::Value as an intermediate value), but once that’s done I can setup an example that loads the prefab as a RON file for regression test.

That said, I do personally prefer JSON and feel like it would be a better format to use if we’re not expecting people to write prefab files by hand >.>

Ah, I get what you mean. Yeah, I should be able to not depend directly on the amethyst crate, that way amethyst can (eventually) re-export amethyst-dynamic-prefab.

Nah, I saw your responses. Sorry for being slow to respond in turn, I just got distracted while writing up my response :sweat_smile:

2 Likes

(Jacob Kiesel) #6

I’d like to start a working group for pulling this past the prototyping phase, settling on an agreed upon API, and integrating it into the engine proper. @LucioFranco indicated interest in participating in this. Can we transfer this repo to the amethyst org?

I for one would like to start with the Box<Any> approach as it seems more flexible to me. What would you say the merits of the approach you’ve currently implemented are?

2 Likes

(David LeGare) #7

I’m in favor of moving it into the amethyst org and handing it over to the assets team for further development. I’m also totally in favor of switching to the Box<Any> approach and serde_dyn, though I’d recommend removing TypeUuid from serde_dyn and using the type-uuid crate that is in the amethyst-dynamic-prefab repo. Speaking of which, it might also make sense to move type-uuid into its own repo (and I’m fine with the Amethyst org owning that one as well).

2 Likes

(Jacob Kiesel) #8

Since type-uuid is more general it might be appropriate to keep that out of the amethyst org, though I’m not sure where. Do you mind hanging onto it?

0 Likes

(Jacob Kiesel) #9

By the way I’ll be happy to move serde_dyn to use type_uuid as soon as type_uuid goes up to crates.

0 Likes

(David LeGare) #10

Sure thing! I’ll split it off into its own repo and get it published on crates :+1: Once it’s in it’s own repo I’ll also transfer ownership of the dynamic prefab repo.

1 Like