In the one-page letter he claimed the Birkenhead Constituency Labour Party was now governed by "a culture of intolerance, nastiness and intimidation".
Corbyn has conceded that his party has a "real problem" with anti-Semitism in a recent newspaper article.
Frank Field resigns from Labour party saying it is is now a "force for antisemitism".
Meanwhile, senior Labour backbencher John Mann demanded that Jewish MPs are automatically reselected for Labour instead of facing deselection by hard Left activists for speaking out on anti-Semitism.
He said he disagreed with the leader's "blind support" for social governments elsewhere in the world, but believed his domestic policies could be "shaped" to give a voice to those affected by "social and economic upheaval" in the UK.
Talking to Julia Hartley-Brewer on talkRADIO, Chilton said: "If Labour were to adopt their [People's Vote] policy we would really be in severe trouble in many of those leave constituency because, quite frankly, the British people don't like being told what to do and they don't like being told they're wrong, if Labour does that it will change the effect of the ballot box".
Field came under fire after defying the party's leadership and voting with the government on key Brexit legislation.
On Thursday, he denied he was "jumping before being pushed" and used the article to further condemn some local members.
The politician, who has represented Birkenhead for almost 40 years, was told by Nick Brown that he would be removed from the party unless he reversed his decision to resign the whip - effectively the rules and policy an MP is bound by.
Mr Field concluded the article by calling for two "major changes".
Manchester United bully Burnley to prove Mourinho can still bring grit Premier League
During a tour of Africa, Mrs May said: "Anti-Semitism is racism".
One of the local branch secretaries also said Field "refuses" to attend party meetings. His support for Brexit has also put him at odds with many Labour colleagues.
Labour's chief whip has told Frank Field he will be kicked out of the party in two weeks unless he withdraws his resignation from the party whip.
He will continue sitting as an independent MP and says he is considering triggering a by-election in his Birkenhead constituency.
The rebel MP said he will now serve as an independent as he does not want to be seen as part of a "racist" party.
But Corbyn-supporting members of Labour have criticised Mr Field's decision.
"He's obviously lost the confidence of his members and he's now getting his excuses in, it seems to me, and throwing around grotesque slurs, which have no basis in reality", said MP Chris Williamson.
The veteran MP explained to Nick Brown that a "specific bullying issue" also contributed to his decision as the party had failed to take "decisive action" following "numerous complaints".
After he announced he was quitting the whip, critics were quick to point to some of Field's more rightwing views; citing his comments about the need for controls on migrant workers and his admiration for Margaret Thatcher, who he said was "certainly a hero".
Others, including shadow justice secretary Richard Burgon and Liverpool Walton MP Dan Carden, demanded Field move to trigger a by-election.More news: Twitch Announces the End to Ad-Free Viewing for Prime Members
More news: All the latest from the Crazy Rich Asians Singapore red carpet
More news: Modric is player of the year over Ronaldo, Salah