Your design lacks "structure" ....
You may benefit from making a small model from 3 pieces of cardboard, two identical ones for the ends and one twice that size for the top, all taped together to form a upside down U. You will see there is no strength in the joints that are just taped together which act like hinges and allow the ends to fold over. When a design relies solely on fasteners for structure it will fail over time.
By placing triangular braces between the ends and the top, the structure will be greatly increased. The lower down on the ends the braces are attached, the greater the structural integrity. But this interferes with your legs getting under the desk top, so it's not practical. By increasing the width of the frame all around the top, you will increase the rigidity and resistance to "racking".
Braces can be hidden from view like this:
The answer to your question will only be as detailed and specific as the question is detailed and specific. Good questions also include a sketch or a photo that illustrates your issue. (:< D)