If geometric terminology puzzles you, concave objects are the ones that have at least one indentation, a notch, or a bump inwards. Convex objects do not have such inward indentations.
Notice that the original BP #4 had a small square in place of the potato-like object at second box of top-line, left. When I presented the original problem to a friend of mine, the answer was unexpected. This friend found as a "rule" that all boxes at left contain simple geometric objects, treated by elementary geometry (such as a square, an ellipse, a triangle, etc. -- maybe the last one can be seen as a pentagon). Since this could arguably count as a rule, I changed the small square to a potato-like object, to eliminate the ambiguity.
Another interesting note: There is a simple definition in geometry about convexity/concavity: Convex shapes are the ones which, no matter which straight side you choose to augment indefinitely on the plane, this side will never intersect the shape (or, use a tangent line if the shape has curves). This is not true for concave shapes: there is at least one straight side (or tangent to a curved side) which will intersect the shape if augmented in one direction or another.
This simple definition, however, seems not to occur to people when they notice convex and concave shapes (unless they are trained in geometry). If you ask people why is a shape called convex, they will usually not be in a position to formulate an answer, while for concave shapes they may say something like "it has indentations, or bumps, pointing inwards". It seems to me that we tend to observe bumps, spikes, notches, etc., as a whole, without the need to discover and apply the elegant geometric definition -- if we do not already know it.