Modan Publishing House says it won't participate in a cultural boycott of any kind; East Jerusalem sellers report interest in Irish author surging after her criticism of Israel