Justin and Hailey Bieber are spending a celebratory weekend together, following the July 11 release of 'Swag,' Justin's first solo album in more than four years. A source familiar with the creation of the album tells PEOPLE that the Rhode founder supported her husband throughout the entire process.

On Justin Bieber’s surprise seventh studio album Swag, the 31-year-old singer got vulnerable about his struggles in his “Therapy Session” interlude and other songs.

The Rhode founder cozied up in a yellow robe from SKYLRK, her husband's brand in the making, after soaking up the sun in a designer bikini

The insider tells PEOPLE that Hailey "loved having Justin there" to support her beauty brand's big international event