I assume you are asking for tax purposes (that is, that your sister would like to claim her kids as dependents and get the additional exemptions on her tax return). If that's the case, she must meet 5 tests, the most important of which is that she must have provided more than half of their support. Generally speaking, this is not a big issue with the IRS - unless both parents claim the exemption. In the case of divorced parents, what the IRS cares about is that only one parent get the deduction. If your ex-brother in law doesn't claim the kids, your sisters should have no problem doing so. But, if he wants to claim them, then there is potentially a problem since only one of them can get that benefit.
The IRS definition of a dependent requires that all five of the following dependency tests be met:
Member of Household or Relationship Test. At least one of the following must be true:
The dependent lived with the taxpayer for the entire year as a member of the taxpayer's household, except for temporary absences. Temporary absences include attending school, taking vacations, business trips, military service, and hospital stays. (If the person is placed in a nursing home for an indefinite period of time to receive constant medical care, the absence is considered temporary.) The relationship between the taxpayer and the dependent must not violate local laws (e.g., zoning restrictions on the number of unrelated persons living together).
The dependent is related to the taxpayer in one of the following ways: child, parent, brother/sister, stepparent, stepchild, stepbrother/stepsister, half brother/half sister, grandparent, grandchild, son-in-law/daughter-in-law, mother-in-law/father-in-law, brother-in-law/sister-in-law. Also, if related by blood, relatives can include uncle/aunt and niece/nephew. Cousins do NOT meet the relationship test. Relationships established by marriage are not ended by death or divorce. Relatives do not have to be members of the taxpayer's household for the entire year. (There are special rules for children born during the year, adopted children, and foster children.)
Citizen or Resident Test. The dependent must be, for some part of the year, a US citizen or resident, or a resident of Canada or Mexico. Foreign students who stay with you as part of an international education exchange program generally do not qualify as dependents.
Joint Return Test. The dependent must be unmarried, married but not filing a joint return, or married filing a joint return only to claim a refund of withheld tax (neither the dependent nor spouse may claim personal exemptions on the joint return).
Gross Income Test. The gross taxable income of the dependent (all taxable income including money, property and services, unemployment compensation and certain scholarships, but not welfare benefits and not nontaxable Social Security benefits) may not exceed the exemption amount. In 2004 the exemption amount was $3,100. This test does not apply if the dependent is a child of the taxpayer and either under age 19 at the end of the year, or a full-time student under age 24 at the end of the year.
Support Test. The taxpayer must have provided more than half of the dependent's total support for the entire year. (Starting in 2005, the 50% support test only applies to qualifying relatives. For qualifying children, it is sufficient that the child not have provided more than half his/her own support.) Support includes food, clothing, shelter, education, medical and dental care, recreation, and transportation; as well as welfare, food stamps, and housing provided by the state. You must compare the dollar value of the support provided by the taxpayer with the total support the dependent received from all sources. (Note: There are special rules for dependents who receive support from multiple sources and for children of divorced or separated parents.) The support test considers all income, not just taxable income.