Properties by which pixels might be distinguished:
- Position
- Color
- Gradient
- Curvature
and (here's the punchline)
- Whether neighbouring pixels are in the set (ie floodfill)
Define a distance metric on these parameters, eg (difference in color)^2 + (difference in gradient)^2 + etc. A pixel is a member of the set if the nearest example (as measured by the metric) is closer than the nearest counter-example. If membership of the set is a parameter, this selection process would have to be iterated until it converged.