When we try to fit the gymnosperms into the traditional classification system consisting of phylums, classes, orders and such, we get bogged down. Current thinking is that the living gymnosperms form a natural grouping known as the Acrogymnospermae, which is a "clade" with no formal recognition in the traditional system. You can see what's included in the clade Acrogymnospermae (using technical names) at the NCBI Taxonomy Browser.
A "cladogram" showing how different parts of the gymnosperm branch of the Tree of Life seem to have arisen and are related to one another (again, technical names) can be viewed at The Gymnosperm Database.
For most of us backyard naturalists, it's enough to keep in mind that these six commonly known plant families -- out of twelve surviving today -- are considered to be gymnosperms:
The Canada Hemlock tree, Tsuga canadensis, at the right is typical of many gymnosperms in that it produces a woody, brown, seed-producing cone from a female flower-like item, a pistillate strobilus. Such a strobilus appears at the upper right in the picture, looking like a tiny brown pineapple with yellowish golfballs clustered atop it. An immature, blue-green cone is developing just to the left of the strobilus. The Canada Hemlock's leaves are more or less needlelike and are evergreen, also typical of many gymnosperm species.
Some gymnosperm leaves are scale-like, as seen on the Ashe Juniper, Juniperus ashei, at the left. The three brown cones in that pictures are pollen producing male cones. Mature female juniper cones look like fleshy berries, like those at the right.
Gymnosperms are not gymnosperms because many of them bear evergreen needles or scales, and cones. Other plants beside gymnosperms share those features. Club mosses and horsetails produce cones, for example. Leafy liverworts have leaf-like items often referred to as scales. What could be more needle-like that the spine of a cactus?
Here's what makes gymnosperms gymnosperms:
Those are diagrams of immature and mature pine cones, cut down the middle lengthwise. What's to notice is that what's labeled as the ovule in the female strobilus, as well as the seed (which started out as an ovule) in the cone, are lying uncovered, or naked, atop scales. Naked ovules and seeds -- not protected by any kind of covering -- are what make gymnosperms gymnosperms.
At the right you see the mature cone of a Loblolly Pine, Pinus taeda. Once the cone was mature and dried out, its scales pulled apart, and seeds with papery wings began falling out. Three such seeds appear in the picture. Their wings, which help them disseminate on the wind, are part of the seed coat, not anything that had surrounded the naked ovule.
At the left we see the red, juicy, fruit-like items of a Yew bush, genus Taxus. The juicy, red parts are arils, which are fleshy partial coverings on certain kinds of seeds, including seeds of various flowering plants, not just gymnosperms. Arils develop from the ovule's funiculus, which is the stalk attaching the ovule, and later the seed, to the rest of the plant. It's like the placenta connecting the developing embryo to the mother in us mammals. Note that the funiculus is not part of the ovule but rather something else entirely. Technically, the yew's aril is considered to have evolved from a primitive cone scale. The green item nested inside the cup-shaped aril is the seed, partly but not entirely covered by the aril. At the top, part of the aril has been removed to show the seed better. So, naked seed = gymnosperm.