Aphorio v0.6.3

Performance

A primary design pillar of this application is high performance at all stages. This application reports the output of performance timers in many locations, because performance improvements are unattainable without qualified evidence of current operating status.

Performance improvements are not just about executing faster. Improvments to performance also increases execution stability resulting in an application that crashes less even under high load of multiple simultaneous operations.

This application primarily focuses on performance in a few key areas:

Transmission Speed

As of Aphorio v0.6.2 the application is reporting a localhost message send only rate of about 6,000,000 WebSocket messages per second or about 5,000,000 messages per second roundtrip with a response payload. These tests were conducted on hardware with a single AMD 9 9900X CPU and 32gb 5200mhz memory. This is dramatically faster than popular TypeScript based messaging applications Bun and WS which report send speeds of about 165,000 messages per second and 100,000 messages per second respectively.

The key to performing faster is to understand that JavaScript/TypeScript language platform is extremely high level and the involved CPU load is infinitesimal small. This means the performance concerns are almost exclusively, about 99.96%, due to speed of memory operations.

The divergence between CPU cost and memory cost in performing network transmission operations was discovered by accident. While attempting to write a performance profiler to measure message send speed it was discovered that the network bus could process all messages in its queue and drain its buffer from memory as fast as 3,250,000 messages in about 0.002 seconds. This number was validated by Claude AI, but it turned out to be incomplete. Claude and I made the same identical mistake. We did not realize messages were pushed from a queue in memory to the CPU, from the CPU to the network buffer, that network buffer was cleared, and a callback was called all faster than JavaScript could call the network send function from a regular JavaScript loop. The profiler was adjusted to validate the message send loop completed its message send quantity before trusting lower level hardware operations, which reduced the message send speed to about 100,000 messages per second.

Upon discovering message send speed from a high level language, like JavaScript, is almost exclusively tied to language instruction and memory operations the proper path forward became more clear.

The first performance breakthrough came from adjusting message queue handling. The application would push new messages to the end of an array would write to the network buffer from the first index of that array, which would have to be cleared by an array operation to remove the first index. This is expensive, because each time it executes all remaining messages in queue shift forward one index. Instead, the code was changed to push messages onto an array, but an index of the current read position is now stored as a property on the given network socket. Once the network has read all messages in queue the queue array is reaasigned to a new empty array and the queue index is reset to 0. All messages continue to write to the network interface in the proper order but now without adjustments to the corresponding queue array. This pushed message send performance from 100,000 messages per second to 800,000 messages per scond.

The second performance breakthrough came from pushing each send operation into a new call stack. This is counter-intuitive in that it greatly expands the memory footprint and should result in increased time due to the increased instruction overhead from so many stacks, however it increased performance from 800,000 message sends per second to 6,000,000 per second. Strangely, it was also discovered that externalizing from the message send function this push into a new call stack, as opposed to performing that internally to the function, was about 20% faster and more stable.

Aphorio runs on both Node.js and Bun platforms. Performance testing has revealed about 40% faster message send speed executing on Node.js but Bun is overwhelmingly more memory stable than Node.js. Node.js would crash about a third of the time when running a 6,000,000 message experiment of 10 tests. In Bun I could execute a 10 test experiment of 50,000,000 messages. That 50,000,000 message experiment took about 74 minutes to execute, but it still finished. Although Bun is overwhelming more memory stable at high transmission speeds I have still gotten Bun to crash once.

Application Start Time

During start up the application performs a variety of separated tasks to become ready. Many of these tasks involve gathering information OS specific data via command shell, and these tasks tend provide more information on Windows as compared to Linux but take about 10 times longer to run. Other start up tasks include gathering information from Docker, reading configuration data, launch a web server for the dashboard utility, and more.

Each start up task prints its completion time to the shell's stdout for performance review after multiple repeated application launches.

Service Response Time

The Aphorio Dashboard reports service response times in most feature sections. In most cases the time to update requested information typically runs about 0.002 - 0.02 seconds for localhost calls depending upon the information requested. These response times will be largely dependent upon the speed of network message processing and speed of a given service function to execute.

Dashboard UI Performance in the Web Browser

The best means to achieve high performance in a web browser is to respect the compile target. The compile target of the web browser is the document object model. The more abstracted access to the DOM becomes the slower the web browser executes, and that slowness has compounding effects that amplify across multiple operations. A good compromise between DOM abstraction and high performance is to access the desired DOM location once early in the application load once and assign those access points to a JavaScript variable for later reference.

The other large area of improvement is to minimize the number of HTTP requests a page must make. In the case of the Aphorio dashboard UI there are a total of 3 requests. The first of those requests is the dashboard HTML page that loads in the web browser. All required artifacts, such as CSS and JavaScript, are included within that one file. The other two requests are each WebSocket handshake requests. This results in a slower than typical initial load step for the page since there is no incremental loading of the page, but a faster total page load. Despite the Aphorio being a large single page application with a tremendous amount of JavaScript code the page typically completes loading in about 150ms on localhost.