No proper solution

Up today (2021-03-13), there is no immediate way to download a specified folder from iCloud.
The download button is grayed/disable for folders in iCloud.com.
Below there are only hacks.

The evident (and painfully) approach

You can select all the files inside the desired folder and click the download button, but since it does not include subfolders, you must patiently repeat the process for each folder in the tree.

Request Apple a copy of your data

Go to https://privacy.apple.com/ and request to Apple a copy of your iCloud files.
Unfortunately, you cannot select which folder you want to include; you will receive a link to download the whole drive.
They say it could take a few days to receive the link. However, in my experience, it only takes a day.

You can also use this trick to download other data that Apple makes it difficult to obtain in a standard format, such as the Health App data.

The iPhone solution (the best approach)

So, the original question request a solution “without having to sync the entire content of my iCloud Drive with my computer”.
I had the same problem. I was not using iCloud Drive in my Mac, but it was active (probably by default) on my iPhone.
Then, I could use the File app (the built-in App) to copy the whole folder to the internal storage of my iPhone or to other cloud (Google Drive, Dropbox, OneDrive, etc.).

1. Ensure your other cloud is active within the File App.
   Select (...) > Edit in the upper right corner of the File application initial view to activate the available cloud accounts.

2. Browse to the folder you want to copy from iCloud.

3. Long press on the folder name to activate further option, then select Compress to create a zip file of the folder.
   (You can skip this step, but I recommend it if you are planning to copy to another cloud and not for the internal iPhone storage)

4. Long press on the zipped folder, then select Copy or Share.

5. Select a destination for your zipped folder.

6. Wait. It could take longer to copy from one cloud to the other.

7. Use the other cloud to download your files.

8. Complaint to Apple!

Complaint to Apple

Meanwhile, complain to Apple about this unbelievable omission
https://www.apple.com/feedback/icloud.html