The importance of topological and directional relationships between spatial objects has been stressed in different fields, notably in Geographic Information Systems (GIS). In an earlier work, we introduced the notion of the F-histogram, a generic quantitative representation of the relative position between two 2D objects, and showed that it can be of great use in understanding the spatial organization of regions in images. Here, we illustrate that the F-histogram constitutes a valuable tool for extracting directional and topological relationship information. The considered objects are not necessarily convex and their geometry is not approximated through, e.g., Minimum Bounding Rectangles (MBRs). The F-histograms introduced in this chapter are coupled with Allen’s temporal relationships based on fuzzy set theory. Allen’s relationships are commonly extended into the spatial domain for GIS purposes, and fuzzy set theoretic approaches are widely used to handle imprecision and achieve robustness in spatial analysis. For any direction in the plane, the F-histograms define a fuzzy 13-partition of the set of all object pairs, and each class of the partition corresponds to an Allen relation. Lots of directional and topological relationship information as well as different levels of refinements can be easily obtained from this approach, in a computationally tractable way.