Maybe it is an apocryphal story, but I heard that the theoretical chemists got jealous and so started the Telluride Science Research Center.
This (northern) summer I was privileged to spend time at both, and so I offer some friendly comparisons. Both are excellent and so if you have opportunity to attend either, I would encourage you to.
This is highly selective and mostly restricted to faculty, with a few postdocs. Workshops are small, with typically only about twenty participants. For Telluride you have to be invited and for Aspen you apply and are then selected.
For Telluride most workshops run for 5 days. For Aspen they run for 3-4 weeks and participants must come for a minimum of two weeks. Apparently, in the good old days people used to stay for longer
For Telluride this is closer to a small conference with many talks during each day; although, some mornings or afternoons, and sometimes whole days are free. In contrast, in Aspen there are usually at most a couple of hours of talks, and sometimes none, on each day. The emphasis is really on informal interactions.
In both cases this is arranged by the Center. In Aspen it is subsidised by an (NSF grant and so more affordable (e.g. $75 per week for "bachelor" housing = shared apartment).
Both Centers are run by very professional staff who take care of all the logistics. So, organisers sole responsibility is selecting participants and setting the program. Thus, if you want to organise a small workshop this is a very easy way to do it.
Aspen has their own building with offices, so all participants have a desk in a shared office. Telluride meets in a local school and there are no desks for participants, which is fine since the programs are so busy.
Powerpoint vs. blackboards.
Something unique about Aspen is that most talks are on a blackboard. Generally, only experimentalists are allowed to use powerpoint. I really think this is a very positive thing as it significantly increases clarity and focuses on the key points.
Although it is spectacular in both towns, I do think that Telluride is superior, because you can see massive snow covered peaks from within the town.
Again Telluride wins. The gondola is free. Most days I take it to the top of the mountain just to bask in the views. In Aspen I have never taken the gondola because I am too cheap...
In both towns there are nice short hikes literally from the town. Both have trails along the river running through the town. However, for Telluride there are serious hikes you can do starting from the town or the top of the gondola. For Aspen, you have to drive out of town or pay to get the bus to Maroon Bells, which takes about an hour.
Both towns are above 8,000 feet and so this is not unusual. It is strange that I have been to Telluride six times but never had a problem, but my last two times in Aspen I did have had a mild case. One important preventative measure is to drink lots of water.
Travel and accessibility.
The scenic locations in the Rocky mountains come with a cost. Neither is easy or cheap to get to. For both, one may have to fly through Denver, where flight delays and missed connections are not unusual. Some participants drive from Denver.
Both Centers run regular lectures during one evening throughout the summer, given by some participant. These are often quite well attended by the local community or tourists. Given the demographics of both towns (the rich and powerful) I think this is a wise investment. You never know if there will be the next Moore, Gates, or Kavli in the audience...
In Aspen there is a weekly colloquium, given by someone from one of the current workshops, that all participants are required to attend, in the hope of encouraging interaction between workshops. In the past two weeks I heard two excellent talks on biological physics, by K.C. Huang and Lucy Colwell.
Telluride does not do this. Maybe it should.
Physics versus chemistry.
Most of the Telluride workshops are on chemistry or biology, with a smattering on materials science, involving physicists. As far as I am aware Aspen doesn't do much to encourage interactions between physics and chemistry. I think both Centers could benefit from trying to facilitate this more.