Improving a React / Flux event sourced application
Overview
I recently completed a project for a client to improve the performance of React / Flux application.
This particular application is tied to an event-sourced backend and has started to experience lots of odd behavior regarding rapid user input. The problem stems from the way previous developers had integrated update notifications to the application to the Flux dispatcher and the store.
The problems this presented for the client was that edit users were making would dissappear or be overridden by other inputs, causing some confusion and lots of bug reports for developers.
Like I said, the application uses Flux and the data flow as advertised looks like the following:
This is all well and good for basic apps but it doesn't do a good job addressing what the majority of developers use Flux for.
There is almost certainly a module making XHR calls that will need to integrate their responses into the Flux store. When I look at the data flow above, I don't see an immediate way async calls integrate to the store.
You really have to know as a developer, any XHR action is going to be the starting point for an action that will send data to the store.
Application Concepts
Since this is an event sourced application, the data flow is slightly more complex.
Here's a revised mental model of how I see the Flux application working with the current event-sourced backend:
The app itself is split into 4 panes, each one it seems are tied to a base class implementing an event-source connection to a NodeJS backend.
The application state was maintained by the Flux store which was constantly hydrated by a JSON projection provided by the event-sourced backend.
Data from the inputs is persisted to the Flux store as well as sent to the backend. This is where the problems started to occur for the client and I could see lots of work had been to sort of patch up the system, which ultimately just made it even more buggy.
The problem with how the application was designed was UI updates were made in the Flux store but also a similar update was submitted to a backend command service causing a projection to update and signaling the UI fetch a new copy of the projection JSON.
There's several issues with this. If the update to the Flux store isn't done exactly the same way that the projection will be updated, then the UI will look different when the request to fetch the projection completes and the UI is updated, resulting in data seeming like it's being overridden.
A bigger issue is since there are several areas across the application that will trigger an update, there could be a ton of requests going to the backend all triggering UI updates. If a version of the projection comes back that was created before the user continued to carry on through the UI making updates, it will again seem like data is being overridden.
Basically, the application is one big race condition and the user is forced to interact very slowly to prevent losing information to numerous writes being done in the Flux store.
A better Event-Sourced Flux application
What I'm proposing instead of just blindly overwriting the store on each projection update, is that the Flux store keeps track of a stream revision, and only overwrites the projection data if a newer projection stream revision has been received.
Also, since multiple commands might be triggered by the user before the browser has a chance to submit them to the backend, the Flux application will queue them and process them in the order they are received. These queued commands will also be retained as long as their stream revisions are newer than what's in the store, and their contents will be folded into the app data to prevent any flicker.
With these thoughts in mind, here is a proposed diagram of new systems to be designed that will fix the overwriting behavior and also allow the application to operate faster and more effectively for the client.
Command Dispatching
A major concept the existing system implements is deduplicating backend requests. Deduplication is needed since mostly all inputs are tied to a backend command each time any keypress is received by the browser.
In order to prevent a massive amount of commands being sent to the backend system when a user is typing long strings like names or sentences, similar commands get deduplicated so that only the most recent one is sent to the backend either before a different command is executed or a debounce timer expires.
Once the commands are deduplicated, they'll placed into a future command queue, where an XHR library will process them one at a time in the order they were created. Really, this is just a list of commands sorted by an instant field of when each was added to the queue, oldest first.
Flux Store
Since data in Flux JS lives in the store and is read by a root React component, I need to combine the existing store state along with all of the future command bodies, the executed commands, and the body of the currently executing command. This is a departure from the standard Flux mental model, but is a core principle of how this system will work.
Also I'll have to be sure to only apply the bodies of future commands whose stream revision number is greater than the projection stream revision currently saved in the store, since this is what is saved each time the application loads and whenever a new projection is observed.
As you can see in the proposed layout above, the new Flux store is comprised of the three collections of commands, the current projection body / stream revision, and the current app data.
On every emit action, the store will first carry out any view action received by the dispatcher. Once that has completed and before the onChange event has been sent to the UI, all of the commands are folded into the current app data to produce the next data that will be saved to the store. Once this is done, an onChange signal will finally be emitted.
The fold method will do its best to apply command bodies against store data, however it won't try and duplicate business logic and simply modify the store in a way that the UI will know it needs to render a temporary state until the onChange signal is fired.
React Component Redesign
The other major difference in how the application is design is the React components themselves.
Components qualifying for this design change are any components containing data coming directly from or attributing to the projection body. These are fields such as select lists, textareas, text inputs, radio buttons, checkboxes etc.
Since the application is on React 16, all qualifying components must be implemented as a class for state initialization and managing updates.
The data for all inputs will be changed to come from the state instead of being passed as a prop connected back to the Flux store. This is against what Flux recommends, however I don't think the Flux authors had event sourced applications in mind when they wrote the library. Whenever a componentDidUpdate lifecycle method is called, the component checks to see if props have changed and if so updates the component state using setState.
To record inputs, a setState call is fired on every input field interaction and on completion of component state update, the update command is dispatched to the backend to begin the process to update the projection.
Going through all of the qualifying React components, I was able to get rid of a massive amount of onBlur or onKeyUp handlers triggering the backend commands and instead just use the onChange / onClick events. This kept all of the logic and functionality of updating the backend out of the UI layer.
Here is what an example component looked like in the old way of writing Flux JS React components:
const AccountContactName = ({
contactFirstName,
contactLastName,
isShowOrderedByPhoneNumberVisible,
onChangeInput,
onSetData,
orderedByPhoneNumber,
validation
}) => (
<div style={{ display: 'table', width: '100%', borderCollapse: 'collapse' }}>
{
isShowOrderedByPhoneNumberVisible
? (
<div>
<div style={{ display: 'table-row' }}>
<div style={{ display: 'table-cell', paddingRight: '15px' }}>
<CZLabel label='Ordered by Phone' />
<CZInputText
error={!validation.accountSelection.accountCompanyPhone.isValid}
onChange={e => onChangeInput('orderedByPhoneNumber',
PhoneNumberFormatter(e.target.value)
.formatNumber('E164')
)}
onKeyUp={onSetData}
value={PhoneNumberUtils.formatForInput(orderedByPhoneNumber)}
/>
</div>
</div>
<div style={{ height: 20 }} />
</div>
)
: false
}
<div style={{ display: 'table-row' }}>
<div style={{ display: 'table-cell', paddingRight: 15 }}>
<CZLabel label='Ordered By First Name' />
<CZInputText
error={!validation.accountSelection.accountContactFirstName.isValid}
onChange={e => onChangeInput('firstName', e.target.value)}
onKeyUp={onSetData}
value={contactFirstName}
/>
</div>
<div style={{ display: 'table-cell', paddingLeft: 15 }}>
<CZLabel label='Ordered By Last Name' />
<CZInputText
error={!validation.accountSelection.accountContactLastName.isValid}
onChange={e => onChangeInput('lastName', e.target.value)}
onKeyUp={onSetData}
value={contactLastName}
/>
</div>
</div>
</div>
)
Now, with the best practice of writing React components to integrate with an event-sourced Flux JS system (or Redux etc), components now look like the below.
export default class AccountContactName extends React.Component {
constructor (props) {
super()
this.state = {
contactFirstName: props.contactFirstName,
contactLastName: props.contactLastName,
orderedByPhoneNumber: props.orderedByPhoneNumber
}
}
componentDidUpdate (prevProps) {
const p = this.props
if(p.contactFirstName !== prevProps.contactFirstName
|| p.contactLastName !== prevProps.contactLastName
|| p.orderedByPhoneNumber !== prevProps.orderedByPhoneNumber
) {
this.setState({
contactFirstName: p.contactFirstName,
contactLastName: p.contactLastName,
orderedByPhoneNumber: p.orderedByPhoneNumber
})
}
}
render () {
const p = this.props
return (
<div style={{ display: 'table', width: '100%', borderCollapse: 'collapse' }}>
{
p.isShowOrderedByPhoneNumberVisible
? (
<div>
<div style={{ display: 'table-row' }}>
<div style={{ display: 'table-cell', paddingRight: '15px' }}>
<CZLabel label='Ordered by Phone' />
<CZInputText
error={!p.validation.accountSelection.accountCompanyPhone.isValid}
onChange={e => {
this.setState({
...this.state,
orderedByPhoneNumber: PhoneNumberFormatter(e.target.value)
.formatNumber('E164')
}, () => {
p.onChangeInput(
this.state.contactFirstName,
this.state.contactLastName,
this.state.orderedByPhoneNumber
)
})
}}
value={PhoneNumberUtils.formatForInput(this.state.orderedByPhoneNumber)}
/>
</div>
</div>
<div style={{ height: 20 }} />
</div>
)
: false
}
<div style={{ display: 'table-row' }}>
<div style={{ display: 'table-cell', paddingRight: 15 }}>
<CZLabel label='Ordered By First Name' />
<CZInputText
error={!p.validation.accountSelection.accountContactFirstName.isValid}
onChange={e => {
this.setState({
...this.state,
contactFirstName: e.target.value
}, () => {
p.onChangeInput(
this.state.contactFirstName,
this.state.contactLastName,
this.state.orderedByPhoneNumber
)
})
}}
value={this.state.contactFirstName}
/>
</div>
<div style={{ display: 'table-cell', paddingLeft: 15 }}>
<CZLabel label='Ordered By Last Name' />
<CZInputText
error={!p.validation.accountSelection.accountContactLastName.isValid}
onChange={e => {
this.setState({
...this.state,
contactLastName: e.target.value
}, () => {
p.onChangeInput(
this.state.contactFirstName,
this.state.contactLastName,
this.state.orderedByPhoneNumber
)
})
}}
value={this.state.contactLastName}
/>
</div>
</div>
</div>
)
}
}