If a state change is going to persist for more than a couple of frames, then it will be worthwhile to move it into a new archetype. If you have transient events that occur within a frame, then you are best served by creating a new entity to represent that event, and then have a system batch process all events of that type and delete the entity. If you have high occupancy of a particular component, then wrapping it in
Option is an entirely feasible alternative.
Having literally hundreds of components on an entity and performing thousands of component addition/removals per frame is absolutely not common or typical in an ECS based game. A typical game’s frame time is dominated by work that takes place inside loops over entities, and only a small percentage of total entities are moved into a new state significant enough to justify a composition change each frame, even if those composition changes were relatively cheap.
If you have a large array of entity data, with a small subset of those entities containing a component you are interested in, there simply is no way to find those entities that is going to be significantly faster than a linear search. At least not without doing some pre-processing work (like sorting the data or building and maintaining an index) - pre-processing work that is not unlike the cost of sorting the entities into archetypes. If fast filtering an iteration performance is important, then pay the cost of that sorting operation. If it is not, then it is difficult to beat the performance of using
Option if you can’t pull the event out into a new temp entity.
In practise, even in the case where you have designed your game around entities with hundreds of potential components, the vast majority of those components will not change within a single frame and each entity likely only has a small number of them at any one time. The majority of these components could be reasonably dynamically added and removed. Many of them can be better represented as transient events as their own entities. The set that you end up keeping permenantly on the entity as options need not be huge.
If legion were to support first-class optional components, by essentially implementing the specs storage and filtering archetecture alongside its current storage system, that would display performance characteristics very similar to
Option<Component> (it would be able to check 64 entities at a time with bitfield logical ands, but is still otherwise ultimately a linear search), with the only major difference being possible memory savings from being able to use a sparse storage implementation. The cost to that would be an incredible increase in internal complexity to legion and a likely performance overhead throughout due to having to frequently check whether these additional storages have been attached to a chunk.