SOLVED: Dynamic Data causes large numbers of AJAX requests that spike CPU to 100%+

Originally posted this here: WIP: Builder Lag: Massive CPU Spikes when working with dynamic data - #79 by danieliser

Fellow plugin dev here so I’m happy to dig into this if needed, but It took a while to try and track this down, still not quite sure I understand the extent of what is happening, but I have narrowed it for all purposes to dynamic data renders.

In the attached videos/screenshots you can see it happen.

  1. Created a counter element.
  2. Added non dynamic data, you can see it call the REST API for render_element I believe, near instant.
  3. Paste a dynamic data echo function that simply returns a static integer as the start or end value for the counter.

I immediately see dozen + AJAX requests, clicking through appears its triggering a resave on nearly ever element to the root, maybe sibling trees too, not sure.

  1. If you then try to edit said {echo:} function calls, it starts making the same dozens of AJAX requests for every keystroke or 2 (live rendering I assume).
  2. Once dynamic props are on the element, you can modify any field not containing dynamic props and the issue doesn’t occur. If you start typing in a field with dynamic values though it goes nuts.

This quickly stacks up hundreds of requests, bringing even an extremely over powered server to a crawl as each of them slowly finishes or 504 timeouts.

First image is showing that it not only maxed out one node, but 3 nodes. Each node is 10 CPU cores (XEON Gold) and 10GB memory, x3.

You can see the entire process recorded in chrome dev tools network tab. Cut the video as it was several minutes to finish all of them successfully. You can see how this would be painful editing experience, as the editor is nearly useless at the time these requests are processing.

Screen Recording 2024 05 30 at 1 19 47 AM

Screen Recording 2024 05 30 at 1 23 18 AM


Here would be my recommendations based on SAAS & WP experience:

  1. Set up your ajax calls to be cancellable.
  • On client side, every request should be stored and abortable. This should be be used in a few ways:
  • On server side, either ensure your disabling ignore_user_abort during your requests, or explicitly use methods to check if the connection is still active during your processes, bailing as soon as you detect its not, do so with things like connection_aborted() or even connection_status() != CONNECTION_NORMAL.
  1. Any time an element is saved, the requests object should be stored in a keyed object/cache, but before any save, you first check if there was a previous request object in the cache for that element, and if its still running, abort it before making another.
  2. Any time an element is moved or deleted, cancel pending requests for that element etc.
  3. Any time the browsers stop button is clicked, or esc pressed when no element is selected, all pending should abort.
  4. Add a debounce or throttle to your AJAX saves in general.

This is how major SAAS systems handle forms such as Remix.run: Network Concurrency Management | Remix

while (TRUE) {
  if(connection_aborted()) {
      break;
  }
  sleep(5);
}
var xhr = $.ajax({
    type: "POST",
    url: "some.php",
    data: "name=John&location=Boston",
    success: function(msg){
       alert( "Data Saved: " + msg );
    }
});

//kill the request
xhr.abort()

Overall we love Bricks, we have pages that not only look great but take less than 50-100kb to load, but the broken ajax in the editor for several versions now has made this an unbearable experience most times.

Please let me know what other useful info might help solve this once and for all, but I think the above are sound solutions either way for ongoing performance improvements.

Reach out to me directly, twitter, email, Post Status slack, here IDC, lets fix this.

7 Likes

Ok so my original bug report stands as a request for major improvements to the AJAX systems.

BUT I have found via New Relic several processes from other plugins that were making each of the AJAX requests take longer than they should by 2-3x. This resulted in a compounding effect.

Again I reitterate the issue I reported still stands, however the immediate pain is slightly alleviated, though I look forward to working with you to solve the AJAX performance issues.

Hi @danieliser,

Thank you for creating a separate thread :slight_smile:

I tried replicating the issue locally with the code you shared here: WIP: Builder Lag: Massive CPU Spikes when working with dynamic data - #83 by danieliser

But on my end pasting {echo:EstViewsAfterSeconds} generates only 1 request: Jam

I wonder why in your case it’s generating dozens of requests :thinking: perhaps it has something to do with your Bricks settings or the other elements on the page. Tough to tell from the video. But that’s definitely not the expected behavior.

If this is a staging environment, could you please share temporary admin access to help@bricksbuilder.io with a link to this forum thread so I can try replicating it on your setup?

@charaf - I’m currently in the process of monitoring with NewRelic, looking for additional code causing pain points, each one we remove drastically reduces time to process each of the AJAX requests, but I am still seeing them.

I’ll respond to the email ticket I made with info to get into a staging site shortly.

Hi @danieliser,

I have uploaded version 1.9.9-beta to your staging site and can no longer reproduce the issue. Could you please test it on your end as well?

While we weren’t aware of this specific bug, the performance enhancements in 1.9.9-beta might have inadvertently resolved it.

1 Like

@charaf happy to test it out. Can I get the actual changelogs for 1.9.9? Curious to see what all is coming.

Hey Daniel,

it’s already there as a public beta. There you go: Bricks 1.9.9 Changelog – Bricks. :smiling_face:

Best,

André

2 Likes

Well done. Looks like its only making one admin-ajax call now.

Curious are you moving towards optimistic UI/UX as well. I saw you fixed sorting lag, but to use it as example, when sorting elements and saving to server, optimistic UI doesn’t wait for the servers success response to place items where they go and free up the editor.

It goes on as if the request will be a success. If that requests promise comes back failed later, error handling would then restore the sort to the previous order and show an error that something went wrong.

On normal basis, the user gets 0 potential lag.

Just my 2 cents on methodologies that might help as you go into the micro optimization of the editor.

Love the product though, even with occasional issues its leaps beyond others.

PS… Any status on “WIP Components”? Was really looking forward to that one, guessing its either sidelined due to these other priorities or ended up being a bigger job to make sure its done right.

Happy to test.

PPS. I run Popup Maker and a few other popular plugins with nearly 1M users, also an extremely dedicated & evangelical user of Bricks… I’m happy to be a sounding board, tester, dedicated bug finder/feature requester etc.

Thank you for checking the beta @danieliser!

The sorting lag & duplicate AJAX call fixes weren’t directly related to the server itself. It was more about Vue’s state management. A bunch of components were unnecessarily re-rendering due to state changes, triggering computations that slowed things down.

Regarding global components, the relevant thread (WIP: Global Components) is the best place to discuss it. No updates I can share from me personally on that front :slight_smile:

1 Like