The present work describes a new approach to integrating structural knowledge into the image classification process. First, a fuzzy classifier produces a fuzzy partition of the image. Then, the defuzzified (crisp) partition is tried to be improved. According to the membership degrees in the fuzzy partition, the system selects a set of pixels and associates a set of candidate classes with each of them. The initial crisp partition is improved by reassigning each selected pixel to one of the classes it may belong to. This is performed by a combinatorial optimization strategy. The aim is to maximize the adequacy between the regions defined by the crisp partition and the structural knowledge which is available. First experiments on synthetic data as well as on simple real data show the applicability of our approach.