A general definition of ableism would refer to the physical and social ways that people with impairments and lived experience of disability are marginalized, excluded, or otherwise prevented from participating in world and accessing their basic rights. In the ABC-CLIO Companion to the Disability Rights Movement (edited by Fred Pelka), the definition of abelism includes "that set of often contradictory stereotypes about people with disabilities that acts as a barrier to keep them from achieving their full potential as equal citizens in society."
One important aspect of the term abelism is the way it highlights social and physical barriers. Often the disadvantage of disability is thought to be linked primarily to the physical or cognitive impairment, something individual and biological. The disability rights movement, academic disability studies, as well as the lived experience of people who identify as disabled instructs us that often the greater barriers and more pervasive and difficult to challenge obstacles are social, systemic, and institutional. In other words, what makes living with a disability difficult is only, or even mostly, that someone can't walk, or doesn't communicate the way others do, but rather the difficulty comes from expectations, arbitrary norms, and limitations imposed by society.
In the context of sexuality ableism is part of the very definition of what is sexual and sexy in most cultures and societies. The dominant construction of sexy as something that is young, thin, white, heterosexual, which denies that anyone who doesn't fit those categories can be sexy, and sometimes should even have the right to have sex, is essentially ableist. As is the notion that what makes someone not an attractive partner is individual and biological, as opposed to heavily constructed by society. Ableism results in the isolation of particular groups of people, and because it is so easy to internalize ablest ideas, the isolation can be self-fulfilling.
Language is tricky, and imprecise, and because words are often used not just to convey information but to put people in their place, keep them down, make them feel bad, and otherwise allow us to do violence or harm to each other, it's important to consider not only a definition of a term, but where it comes from. Definitions of abelism that come from people who are not primarily on the receiving end of it need to be considered in that light, just as definitions of abelism which grow out of communities who have long been marginalized by the experience of abelism. In other words, to some extent, attempting to affirm a single definition of a term like abelism is a problem.
More Definitions and Discussions of Ablism:
Bitch Magazine: What is Ableist Language and Why Should You Care?
The F Word: What Is Disablism?
BBC.com, Ouch!: Disablism ain't the same as racism
Ableism and Ability Ethics and Governance: Ableism and Ability Glossary