2020 Project Direction

(Fletcher) #1

(I originally had this as a non-public post, but decided to open it up to everyone)

Hi everyone,

With the project seems to have slowed down quite a bit. I think this is a good point to review how we do things, current issues, solutions, etc.

I’m going to first give an overview of the legal structure of the project, so everyone knows what constraints we have to operate in.

Amethyst is 501©(3) non-profit headquartered in the US, specifically Redmond, WA. We formed this a bit over a year ago so that we could collect donations, amongst other things. It also gets us a lot of free resources, such as G-Mail.

We have a Board of Directors structure, which originally consisted of 6 people. Some have moved to Emeritus status. The remaining board members are myself, Erlend, and Azriel.

For awhile, I’ve functioned as Executive Director (equivalent to a CEO of a for-profit), in that I handle most of the administrivia and paperwork. I also cover the expenses our donations do not.

ED is an officer position, as opposed to a Board seat. Board members set quarterly/yearly strategy, officers carry it out.

Regarding current status…

From what I can tell, we’ve hit the same problem we used to of lacking project management. Amethyst is big and complex enough that it needs more management, but no one has the time and/or skillset. A few of us have tried, and at least in my case, it burned me out. It’s pretty much all I can do to deal with the paperwork side.

I’ll leave it there for now, as I’d like to hear ideas/thoughts/suggestions on how we should proceed. Pretty much any option is on the table in terms of structure and workflow, so feel free to throw anything out there.

8 Likes
(Khionu Sybiern) #2

Before anything else, I think the switch to Legion needs to either be completed or formally abandoned. That pending change to our core is probably quite the deterrent, for contributors and users alike.

After that, I think we would benefit from a small group that is focused on the product side of project-wide planning. Said group would find all the things that need to be fixed or could be improved, break them up into small units of work, and coordinate their completion, possibly even mentor. This group would limit the amount of development they themselves take on, and focus more on trying to keep contributors engaged and unblocked. This group could also be in charge of the release cycle.

The core of this idea is to make it so units of progress can “have the end in sight”. It’s hard to keep working, especially on an OSS project, if there seems to be no end to what you have to do.

1 Like
(Fletcher) #3

So, how far is the Legion integration, and what is left? Is this another massive rendy-style project that will take over a year?

(Khionu Sybiern) #4

Afaik, it was almost done. I can do a review this week. The work was being done almost exclusively by @jaynus.

(Fletcher) #5

ah, ok, thanks. @azriel91 do you have any additional info on this?

(Azriel Hoh) #6

heya, yeaps. Shall list high level points, and detail is expandable if that point is of interest.

Observations

Amethyst is very appealing, but isn't beginner friendly

i.e. Amethyst’s branding is very attractive, but the software has a high bar to use effectively.

Symptoms:

  • Many people in Discord are interested in trying out Amethyst, but are also Rust beginners.

    Many people ask similar questions (vulkan feature isn’t working, how to do a nested join()), meaning they’re still learning Rust’s crate model / memory model.

This is a good opportunity to improve

We do not have enough consistent active developers

Symptoms:

  • Users’ Github Issues are not addressed (at all)
  • PRs are not merged (in a timely manner)
Amethyst is not contributor friendly (whether member/TC or not)

Symptoms:

  • People want to improve Amethyst, but are stagnated on “is it okay to do this? I may step on someone’s toes.”
  • Code isn’t easy to understand – not enough conventions, too many items in each Rust module, and re-exports seem arbitrary.

Examples:

  • Legion / Atelier Assets
  • No one is brave to change renderer (or if brave enough to change code, maybe not brave enough to make it part of Amethyst)
  • Or maybe it’s just lack of time to change things because they’re too large / complex, or like, life.

Proposals

These proposals attempt to address a number of the aforementioned issues together.

Allow merging PRs from experts who do the detail, even if we lack understanding

Reasons:

  • Gets contributions in, and not stagnate.
  • From people who pay attention to detail, we can trust their code more than our (fuzzy) understanding.
  • Improves the contribution experience, and also our reputation for being active.
Archive `amethyst_tools`, explain starter project in docs instead

Reasons:

  • Not enough people are active / have the time to maintain it, it’s not automated.
  • Rust beginners don’t understand --features "vulkan", and docs give us the opportunity to explain.
Version all member crates with the same number

Reasons:

  • Can automate version bump.
  • Can automate publish from tag.
  • When some downstream crate breaks / releases a fix, we can too within 1 hour, with only 10 minutes human attention required.

See Amethyst Versioning and Release Process for details.

Note: also requires tests to not depend on amethyst itself (cyclic dependency).

Separate all tests into one crate

Reasons:

  • Faster compilation times (only link once).
  • Less disk space used.

These may make it more/less contributor friendly depending on one’s perspective – tests living in a separate location do need you to open that file, but I think good code structure convention + IDE “open this file” addresses those.

See Amethyst Versioning and Release Process for details.

WASM and Legion, pick one

Clarification: Legion and WASM can work together, but given the amount of activity / time we have, it’s better to whole-heart one thing, than half-heart two.

Reasons for Legion:

  • Legion is something people have been waiting for, and is very promising in performance.
  • Atelier-assets is also amazing in terms of its functionality (have tried it myself).

Reasion for WASM:

  • GL support, which also many people ask for.
  • Many existing projects are on specs, and pushing this route would enable those on the web (which may also bring more uptake).
  • MOSS grant.

Personally I’d choose WASM because a non-trivial duration of my life has been spent designing my game on specs’ API, so if we collectively pick the Legion route, then realistically I may fork (as I’m unable to sustain development).

If there’s one direction I’d choose among these proposals, it’s to focus on WASM – the other proposals are naturally part of the work to make it more efficient.

4 Likes
#7

I’m quite new to this project, but already have some insights. I pretty much agree with everything what @azriel91 said (well maybe except test organization, but these are minor issues).

Regarding not being beginner friendly, I find that it’s mostly due to difficult to understand abstractions, that bury everything under many layers of advanced rust code. One of the biggest offenders is prefab system (see https://github.com/amethyst/amethyst/issues/2197). I find myself answering the same questions over and over again because all examples are abstracted behind prefabs, when they should be as explicit as possible. When I started with amethyst I had to jump straight into analysing core code as examples do not help at all. Suggestion: revert https://github.com/amethyst/amethyst/pull/793, fix other examples and add as much comments explaining code as possible. Also it would be good idea to track most commonly asked questions as a list of what examples are missing.

Legion ECS switch is really painful and personally deterred me from investing time into amethyst half a year ago when I noticed that a big change is underway. Even for existing members I think it is a reason not to write any complicated additions, because it’s unclear which ECS will be used. AFAIK, legion switch is still a lot of work (examples, book rewrite), but at least a decission must be made if we switch or not, work can happen later. This would at least let to write code without worrying that it will need to be rewritten in the future.

Personally I think that legion performance gains do not matter for 90%+ games written with amethyst and penalty of misuse (frequent component swaps) just makes it way worse. I don’t see any way this would improve amethyst as an engine, except for benchmark charts. To sum up: a lot of work for little gain, but since most of the work is already done, this makes it a hard decission.

I agree that we should firstly proceed with WASM merge as that is mostly waiting for upstream releases.

3 Likes
(Alve Larsson) #8

Here are my few cents that I find most important to address.

If Legion now is supposedly almost done then getting that out asap is paramount.

Separate all tests into one crate

I like this proposal very much, starting and running the engine is a pain point we must improve on, it was the major factor for my friend not picking up the engine as he were not able to debug and test code quickly.

Regarding the asset system and the examples,
@chemicstry is making a great point here, All the examples are just variations of how to use the asset system and not, for example, create hardcoded objects and entities from scratch, it’s important as it helps to teach how the engine works in a lower level, I think we need to dramatically cut examples that are just an asset system showoff and show how to actually work with the engine in rust.

3 Likes
(Joël Lupien) #9

Oh henlo there friends!
Its opinion time!

yes

yes

yes

yes

yes

Depends. We can do automatic version bumps pretty easily with sed or awk (they are able to increment numbers). So this isn’t strictly required.
This change would make it easier for people to understand that amethyst_assets=“0.16” is in amethyst 0.16.
However, this change also makes it harder to publish new features. Let’s say I make big changes in the ui crate. Now, if I want to publish them, I need to wait until every other module is ready to get published.
Now, you might say that the modules are too dependent to be published as standalone, but I have a solution for this which I will discuss soon.
My conclusion: Its not too clear if we want to do this or not.

My opinion: unit tests should live right next to the code they test while integration tests should live in their own crate (amethyst_test)

I’m not really interested in either of those, but I do see more value to be had with wasm.
Legion is a massive change that doesn’t seem to give much for its drawbacks.

Agreed. It is a limited system in what in can do, it is complex and has a ton of boilerplate.
I agree that the examples should have been written without prefabs.
I don’t think we can yet replace prefabs with anything else, or that its a good idea to remove them, but we can make the learning curve easier.

Most importantly: Have fun and a great day everyone! <3

4 Likes
(Timon) #10

Three points I noticed:

  • Amethyst grew too fast this went well until the management of fletcher fell away.
  • The community is too much divided.
  • It’s too hard to do quick fixes with pull requests.

Community Division
There are several places where the amethyst community is located: discord, forums, github, redit. There are 57 discord channels set up. I think those channels have an upside and a downside. The positive side is that we can do our job without influencing other discussions, the negative side is the result of the positive side. There is little communication between teams and different areas within the project and the few people who already use discord are also distributed.

I think the first thing to do is to focus on getting the community together. So that people can get back in touch with each other and maybe there will be a motivation spark again. This can be done by archiving a lot of inactive, useless, non-project releated discord channels (#languages, #petting-zoo, #lets-game, #books, #game-feed, #bouncer to name a view).

Make it easier to contribute small fixes
Creating a PR for a documentation fix, broken link update should not require half an hour of work (from my experience). I think that the current PR templates ar way too much overhead for such thing. Maybe create a simple version?

Discord poll February 2nd

Dates are of the last message in channel

#general
bot-usage: 6 january

# game-creation
logic: 23 january
art: 5 february
showcase-team: 11 december 2019
game-jam: 22 october 2019


# development
- bikeshed: 22 december 2019
- audio: 5 february
- editor: 18 january
- scripting: 14 january
- tools: 21 january

# internal
- infra-logs: never
- bouncer: 16 juli 2019
- bot-usage: 07 august

# internal offtopic
- offtopic: 01/15/2020
- petting-zoo: 12/20/2019
- books: 10/08/2019
- lets-game: 10/21/2019
- game-feed: 01/26/2019


# languages:
- french: 12/10/2019
- german: 06/16/2019
- japanese: 08/10/2019
- russian: 09/27/2019
- chinese: 11/15/2019

# voice-to-cat
10/05/2019

As you can see a lot of channels are unused for about three months until 2 february.

3 Likes
(Azriel Hoh) #11

Oh ya, a compromise I’m happy to have re:tests is, at least let’s not have cyclic dependencies in Cargo.toml – crate X should not depend on crate Y in dev-dependencies if crate Y depends on crate X.

That at least allows us to publish from a commit without editing Cargo.toml files.

(Fletcher) #12

Do you think there is value in staying on Discord at all?

I agree about pruning unused channels.

(Fletcher) #13

Thanks everyone for the feedback. It sounds like there’s a few major issues that need to be decided, and then streamlining and enabling contributions.

Let’s see what we can do to fix some of this! If anyone is interested in participating in some cleanup/streamlining efforts, please post here.

I’ll make another post as we work through the bigger decisions

(Timon) #14

I can’t really say, it’s quite a radical change which likely will have an effect on the community. Maybe positive or negative.

Rust already has a community discord and a game dev discord. Maybe it isn’t that bad of an idea. If all amethyst related things are discussed there then the rest of the rust community will be indirectly involved. Wich might spark contributions. An other option would be to create a channel in rust game dev.

(Thomas Gillen) #15

I have been spending just about all of my free time (and some time I should have been spending elsewhere) on the experimental branch in legion, which is now close to feature parity with master (and is a significant improvement in both code quality and performance). I was planning on switching my focus soon to working much more closely with @aclysma and @kabergstrom on getting their serialisation and editor work properly integrated, and then moving my attention over to more directly contributing to helping out with Amethyst itself. I am also willing to move the legion repo over to the amethyst org and hand over more control of the project.

I do not really know what would be best for the Amethyst project for the immediate future, though, and understand that a switch to legion is taking longer than expected and is a blocker for other work. Hopefully once I can shift my attention over to more directly assisting with Amethyst things might pick up pace.

@chemicstry I don’t really want to derail this thread here, but the performance cost for inserts and changes is, I think, overblown. For entity insertions, legion’s experimental branch is pretty fast (inserting 10,000 entities with 2 f32 components):

legion-master insert time: [589.54 us 592.71 us 596.27 us]
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high severe

legion-packed-archetypes insert soa
time: [47.888 us 48.051 us 48.266 us]
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe

And I have been doing some experiments around component insertions/removal and found that if you have components that are added and removed at a high frequency (which 90% of the time are when you are using components as state transition flags), using an Option<T> as your component is much faster for both setting/unsetting and iteration speed than actually adding or removing the component in other ECSs. If you have extremely low occupancy and your components are large, you could alternatively store these components side-channel in a hashmap or sparse array resource, which ultimately ends up being very similar to what other ECSs do to acheive faster insertion speeds.

4 Likes
(Khionu Sybiern) #16

Regarding the Discord, if we’re about to do a revitalization of the project, cleaning up the Discord would add to the visibility of that energy. I have been opposed to doing larger structural changes to the Discord until it such a revitalization occurred.

Changes I’m mulling over:

  1. Archiving (public read or serializing/zip) the entire Development category
    • While it is a large step, we’ve wanted to use the forums as the primary conversation platform when it comes to actual development work. The Development category was supposed to be for work on the engine as opposed to consuming the engine, and that doesn’t make a lot of sense to have along side the forums, especially since it attracts a lot of people seeking support.
    • My alternative is to put the whole category behind an “opt into the Development channels” wall, which would be as simple as clicking a Discord Reaction. In that wall, we could explain what the Development channels are for, and encourage people to use the forums instead.
  2. Create more support/discussion channels for consumption, move Game Creation be more for users to showcase their work.
  3. Get rid of the Language Category.
    • As a Verified Community, we have certain moderation obligations. Unless we have at least one person who is taking on moderation responsibilities who knows a given language, we can’t moderate them.
    • Additionally, aside from French, none have gotten more than a handful of messages in the last several months, and even French hasn’t had that much activity.
3 Likes
(Kae) #17

I made atelier-assets for use within Amethyst, but unfortunately the current prefab system loads stuff in-line based on paths, so it wasn’t compatible with atelier-assets. I didn’t want to push for an integration of atelier-assets that couldn’t reach feature parity with what’s already in, so I made a new prefab system for legion. The new prefab system (https://github.com/kabergstrom/prefab/) is similar to Unity’s prefab system in its design. It’s hard to support a prefab system like this in specs because specs doesn’t support working with component types generically using component type IDs and pointers. Specs’ Storage APIs don’t expose this, and this was also a hinderance to accessing data from scripting languages.

4 Likes
(Khionu Sybiern) #18

This is another big point. Others noted that sharing access to Specs-owned content was a complicated matter over FFI.

1 Like
(Azriel Hoh) #19

As long as enough people are active, focusing on Legion is better for the longer term – WASM still needs polishing from the lower level libraries (like, as basic as the wasm-bindgen level).

The risk is that there is hype now that a heavy decision is being made, and after the hype goes away, will the development continue to be consistent? I think it can – let’s make the contribution process easier:

  • Don’t require checkboxes on the PR templates – some are really heavy.
  • Allow 1 person approval merges (given enough experience / educated judgement)
  • Communicate! – simply reply saying “sorry I haven’t got time to review this in the next 2 days, can someone else take a look?”
  • Changes as small as required – WASM had heaps of changes, but none larger than necessary to fix “just the one thing”, and sometimes “half a thing”.

Just a heads up, I wouldn’t be able to work in the capacity that I’ve put in for WASM, but I encourage people to use a similar management process (I see the github project is already created). Oh yes, and make sure you have an end-to-end project – you need to know what the consumer experience is like.

p/s: WASM’s always been hyped, and though heroic efforts pay dividends for discovery, it is consistent persistence that actually made it usable.

4 Likes
#20

As I understand WASM is currently blocked on waiting for upstream releases? I know that there are some other big usability issues (and a lot of them is contributing to upstream), but given the now limited resources, I think it’s worth merging WASM when it unblocks, because it brings a lot to the engine like gl support, updated rendy and solves some old issues. There may be some minor work to ensure that it is in feature parity with master. In the mean time I think we can focus on legion and just merge whatever is finished/unblocked first.

And yes I agree with all the points of making contributions easier. Communication one is really important, we are sure not attracting any new contributors by ignoring any attempts.

Regarding discord: I think it has two sides. Fast communication speeds things up, but also a lot of good conversations and decisions get lost in the history. We should encourage development discussion to happen in github and leave discord just for help (also development help).

1 Like