Webflow doesn't let you control all the elements of a Lightbox directly, but it's easy to address this by targetting the elements directly with CSS Custom Code.
.w-lightbox-close { background-image: url(URL_OF_THE_NEW_IMAGE)!important; }
.w-lightbox-left { background-image: url(URL_OF_THE_NEW_IMAGE)!important; }
.w-lightbox-right { background-image: url(URL_OF_THE_NEW_IMAGE)!important; }
.w-lightbox-backdrop { background: rgba(57, 38, 76, 0.9); }