Joaquin Phoenix Photo by: PA Photos / Landov

After announcing his retirement from acting in October, Joaquin Phoenix made his first public foray into the music world Friday, saying he was ready to put his real self out there and not hide behind movie characters.

"This is me saying this is who I am. This is my story," the actor-turned-aspiring rapper told PEOPLE before taking the stage at Las Vegas club LAVO.

For the skeptics, Phoenix says he's serious about his new line of work. Music, he told PEOPLE, is his real love.

"After all the years of reading scripts and reading lines, this is my chance to do something straight from the heart and put it out there," he said. And why hip-hop? "When I was young I liked punk rock music but then I discovered rap. I love the storytelling aspect of hip-hop."

With brother-in-law Casey Affleck (who's married to Phoenix's sister Summer) capturing the action for a documentary about his journey into the music world, Phoenix took to the stage and performed three songs from his upcoming yet-to-be-titled album.

Not Worried About Critics

Fluttering around the club's catwalk, Phoenix, 34, rapped to the sounds of a drum beat and encouraged the crowd to dance. Though dancing was at a minimum, the crowd watched intently as a scraggly Phoenix - who donned a scraggly beard, long, unkempt hair and a ripped gray hat - hopped up and down and pumped his fist in the air during certain lyrics.

"You hold a very special place in my heart," the Oscar-nominated actor told the crowd near the end of the performance.

While the crowd cheered for the songs, Phoenix said he was prepared for mixed feedback before taking the stage. "Are there people out there who think I'm a joke? I'm sure there will be," he told PEOPLE. "Are there people who think it's going to suck? Probably, but I can't worry about that."

Following the performance, a happy-looking Phoenix spent the rest of the night hugging friends and taking pictures with fans before leaving the club around 1:30 a.m.