I'm just starting out with level design after playing the game regularly since the beginning of September. I mostly agree with Geiss here; playability is the main factor. I do like good visual design, but no matter how good a level looks I'll probably just beat it once and forget it unless it offers a challenge yet is still beatable with effort. Now that I've designed and playtested my own first level, I have a newfound respect for how hard that really is.
Anybody can make an impossible level, but those aren't any fun to play. And not knowing the average skill level out there makes it hard to know when a level is too easy. I think I'm pretty good but I don't know how I rate against others... yet the only way I have to judge a level is how hard it is for me to beat. I really wish players could give designers more feedback. I want to hear what people don't like about my levels, so I can make better ones. A single-axis rating doesn't tell me much.
Aside from general playability, here's what I like to see in a level:
AI motion. There are too many levels out there that look really hard, but that have a "sweet spot" where I can just park, aim at the core, and wait. Sometimes I have to dodge around a bit and spend a few lives clearing a spot, which can be interesting, but once I get that spot the level turns boring. On some large levels I've even locked my mouse button down and left the room, knowing full well I would come back to find the level beaten. Any kind of motion makes this impossible. If for whatever reason you don't want your core to move, make a turret or two that rotate to the ship and put a few single-shot guns on each. (Don't do this with lasers!)
Moving parts. A few moving parts can make a level much more challenging. A long chain of small parts, each with a small angle of rotation, can make a nice flexible tentacle or tail.
A balance of different weapons. Sure, there are some interesting level concepts that contain only lasers or only missiles or whatever, or at least they were interesting before everybody imitated them and turned them into cliches. I'm talking especially of the levels where you have to circle around and around running away from a cloud of missiles to big to shoot, while every rotation you pump a few shots into the mass of launchers. Or the ones that are nothing but a forest of lasers that split the screen into slices so tiny your ship barely fits into them. Using different weapons makes for more interesting levels.
A challenge up to the very end. Unlike Geiss, I actually like the twin-rapid core gun; as a player, it keeps me from getting complacent. I have to decide whether to leave a weapon or two on the boss so I can get up close and hose down the core, or knock off everything and shoot the core from afar as I dodge those darts. A core sitting there defenseless while I pummel it point-blank gets boring after the first ten seconds or so. With the current state of the game, the rapid core gun is almost the only way to make a level interesting at the end. Maybe someday we'll have the option of turning on new weapons as parts of the boss are destroyed, or some such, but so far that doesn't exist.
Visual design. This can be hard, but some people are very good at it. Probably my favorite level is "Scorpio," which I find the ideal combination of visual design and playability.
Some things I don't like to see:
Impossible levels. Do I even need to mention this? Some people make levels that are completely impossible, and they seem to think they've done something clever. No, it's not clever, it just makes you look like a dick. Of course, there are a few levels that look impossible, but actually just require a really high skill level; those aren't so bad. But surrounding the core with forcefields, or putting it offscreen, or having five lasers aiming directly onto the ship, or covering the screen with projectiles and only providing one life-- those are stupid and boring.
Imitations of other levels. Yes, I know everybody's first taste of the editor is when it opens with the level-nine boss in it after beating the main game. But that doesn't mean I have to like all 300 levels that just consist of that boss with a few pieces added. It's even worse when someone copies someone else's original design and puts their own name on it, though that should cease to be a problem now that you can't edit other people's levels anymore. (Granted, I'll miss being able to see how certain clever ideas were done, but I'll live with that if it stops the copiers, I guess.)
Endless spawn death. I really don't like levels where I have a hundred lives but the first ninety are spent frantically dodging around hoping to get three or four shots into the boss before dying. Simply adding lives doesn't make a too-hard level fun; it just makes it into a weary slog.
Lag. If your boss is so big and fires so many guns that it slows the game down to 4 FPS, your level will be annoying instead of fun. At least on my machine, there seems to be a limit on the number of objects on the screen; put enough spreaders on a boss, and the shots will all disappear before the reach the edge of the screen. Occasionally, someone will make a fun level based on this principle; you have to find a way to shoot the core without reducing the number of spreaders to the point where the shots actually reach you. But for all I know, this works differently on different machines, and those levels are interesting only by accident.
Too-large rotating bosses. If part of the boss extends off the edge of the screen, rotating the boss too fast can make the level impossible because the player is always getting swatted out of the air by some part or other. Used cleverly, this can be interesting, but it's hard to do well. This also applies to lasers, actually, though since they aren't on all the time it's easier to make a playable level with rapid rotation.
Too many missiles. Missiles add interest to a level, but there's definitely a point of diminishing returns. Too many missile launchers means the player will never be able to hit the boss unless they have room to move far and fast.
As for the poll topic, I think it's unfair for designers to be able to rate their own levels. Personally, when I see an unrated level, that makes me more likely to play it, not less; I like being the first to add a rating.
I'm really happy that we now have the ability to rate levels without finishing them, because there's some impossible ones out there where it looks like the creator rated them high first, then edited the level so nobody else would ever be able to leave a rating.