Probably going to get battered for this, but IMHO, there should be no such things as holiday lets or "second homes". All they effectively do is put up the prices of property and take housing stock out of the housing stock, if you see what I mean. I have no problem at all with people owning more than one property, but if they do, they should be compelled to rent out the "spares" on a long-term basis.
In lots of the more scenic parts of the country a huge proportion of the houses/flats/whatever are owned by people who leave them empty for large parts of the year, while locals struggle to find anything to rent or buy. Second homes in particular are usually owned by people outside of the area who object to new houses being built and spoiling "their" view, when they are the very people who cause the problem of insufficient housing. If you want to visit an area, do so by all means, but stay in a hotel or bed and breakfast, you'll almost certainly save money and be employing the locals at the same time - win, win. Also of course, as others have pointed out, if you're the landlord, it will probably be less stressful in the long run to have a long-term tenant.