if the part of your body losing the mass was actually completely random, the atoms would disappear roughly evenly across your body, so probably the main thing you have to be worried about is your DNA (and thus getting cancer).

No, if you were just worried about energy, you would be much less tired than by doing things manually. For example, moving a heavy object downhill would gain you energy, not lose it, and keeping a heavy object in the air would neither gain nor lose energy. it would only be lifting that is hard, and it would still be easier than lifting manually.

Edit: and according to Wikipedia, human muscles only have an efficiency of around 20%, so doing basically anything through magic would be 5x easier than doing it by hand.

Obeying all of the laws of physics, magic wouldn’t work at all. Assuming you were able to break a few but kept some such as conservation of energy, it would be very powerful, ex: teleportation does not break conservation of energy as long as you teleport to the same height you left from. If you were able to extract energy from your surroundings, you could probably do basically anything you want. You could lower the sun’s heat by a degree and be able to move a mountain.