TLDR: Their visual cortex evolved in a very different direction to ours
This is going to be extremely speculative, and it doesn't quite meet the specification of incapable of understanding discrete quantities, but it comes the closest I can think of. Further disclaimer: I am not an evolutionary biologist.
Humans see the world like this:
The human visual cortex is very good at a number of complex tasks such as pattern recognition and edge detection. We are very able to quickly process this image and label the main discrete parts of it as the sky, the sun and the sea.
The Continuuoids see the same scene like this:
What has happened here? Well, it turns out that the Continuuoid visual cortex processes information via a Fourier Transform. Effectively, their brain takes the signal from the rods and cones in their eyes, and passes it through some sort of neurological structure that breaks the signal down into a collection of individual sinusoidal-waves. The Continuuoids perceive the amplitude and frequency of these waves, which are all continuous variables.
Why would such a neurological structure evolve? Take a look at the two following star-fields.
If you are looking at these through regular old human-vision, it may take you some time to realise that these images are the same one, just rotated by 90 degrees. Lets look at the same images after they have been passed through a fast fourier transform (FFT):
There is a thin grey line, that is vertical in one picture and horizontal in the next. Effectively the FFT encodes the symmetry of the picture very clearly. This means the orientation of the image is obvious in an FFT. This is very useful if you are trying to navigate by regular features when you are unsure of your own orientation. This is why human engineers use FFTs as part of navigation systems which determine position from star-sightings.
In their evolutionary history, the proto-Continuuoids were a species that faced an evolutionary pressure to navigate long distances accurately by visual sightings alone, and with uncertain orientation. Perhaps they were once flying creatures on a dark world with good star visibility, but strong and unpredictable winds. Perhaps dwindling vegetation caused there to be a pressure to roam further and further afield, while still requiring returning to specific spawning grounds. There are lots of ways this could happen, but this is a somewhat hand-waved way. Through some chance mutation, their existing neurological structures became better at decomposing star-field signals into a limited number of waves. This allowed some of them to find their way back to the spawning grounds more efficiently, and pass on their genes. This is an example of an evolutionary pressure which drove their visual cortex to get better and better at emulating a Fourier transform. This is all wild speculation, with little to back up how possible it could be, but it's the best I can think of.
FFT's have been known about for a long time, and can be performed quickly and efficiently, even using older analogue electronics. I think it is within the bounds of possibility that that a neurological analogue of an FFT processor could evolve.
Although FFT's are used for edge detection, that isn't necessary for species survival. You don't NEED to know the precise boundary between the predator and the forest in order to know that you should run away. You just need to know that something big is getting closer, and the last time your visual cortex picked up that particular cocktail of frequencies, something much like yourself mysteriously vanished.
It seems plausible that if they later experienced the evolutionary pressure to become more intelligent, then the neurology of their visual cortex would influence how their conscious minds developed. They wouldn't see a rock and make it into a tool. They may instead perceive an amount of "rockness", which they can convert into an amount of "toolness". It stretches the bounds of how we can describe it, but it seems reasonable that if they only see in quantities of frequencies, then all they perceive is a mish-mash of different semantic quantities everywhere. They may not even have a fixed boundary of self. They know about the world the most where they are, and they know less about things further away. Therefore, they construct the idea that there is a local maximum of "me-ness" that tails away the further you go.
I would assume that eventually they stumble on the idea of discrete quantities, as it is still a viable way to look at the world. Their equivalent of classical philosophers may play about with the idea, only for it to be put on firmer mathematical footing in their middle ages and early modern times. Their understanding of mathematics develops similarly to our own, but in a very different order. Perhaps infinitesimal calculus was developed in antiquity, but modular arithmetic was only invented much later. Eventually, Discrete Number Theory (DNT) proves a useful conceptual tool, and kids are taut it in Continuuoid high school, complaining about how they will never use this in day-to-day life. Scientists engineers and mathematicians are all aware of DNT, but try to avoid using it if they can, because it's awkward and counter-intuitive to them.
