Hello, thank you for your efforts!

I've been using MRI_reface for an HN dataset and encountered several issues where the tool did not perform as expected. I've detailed these instances in the uploaded document. (I used about 150 HN cases from TCIA but only about 40 were successful. I've tried using both DICOM and Nifti images, cropping the images to include only the head, and changing -altReg to 1 for failed cases.)

Could you also provide more details about the registration methods:

1. 
   

   Your manuscript mentioned that MRI_reface uses ANTs symmetric non-linear registration. What alternative registration method is used when -altReg is set to 1?

   
   
   
2. 
   

   The description of -altReg states "Internally, coreg runs with two different masks, then the one with the lowest cost function is used." Which two masks are referred to here?

   
   
   
3. 
   

   Will MRI_reface return outputs in the original image space when I apply manual external registration and provide the registration file?

   
   
   

Thank you for your time and assistance.

Du,

These are gross failures >90 degrees in the affine registration step. Typically this comes from input .nii files where the orientation information (sform, qform) are not accurate. This may have been caused by your cropping program or any other pre-processing. Typically raw DICOM or DICOM converted with dcm2niix would not have this issue. You can confirm by opening the .nii with an image viewer that shows A/P S/I L/R labels, and verifying that they match the image itself. 

1: ANTs nonlinear is always used regardless, but your failure here is in the initial affine registration, before the nonlinear. reg_aladin from niftyreg is always used for affine registration, unless you provide your own registration parameters file.

2. Two masks are defined in the template space. One prioritizes the face region in the registration cost function, and includes the whole head. It uses the one with the lowest cost function, between the two. altReg=1 tells it to use the result from the mask that had the higher cost function, instead. Rarely, this fixes smaller registration issues, but your issues are coarser than this is likely to fix.

3. You should provide inputs in native space, and an affine registration to the MCALT_FaceTemplate space. You shouldn't resample the inputs to that space ahead of time. If you use native-space input images, the outputs will also be in native space. They will always match the voxel spaces of the input file.

Let me know if you have further questions.
Best regards,
Chris Schwarz

Thank you, Chris, for explaining those details.

In my initial run, I used raw DICOM data, and most of the cases failed. Specifically, in example case 4, the face and ear mask showed very high intensity, and in case 5, the template face was applied to the wrong area. I've since converted the DICOM to NIfTI and cropped the image to focus only on the head.

I've uploaded the original DICOM for one sample case where the defacing result has a similar issue to example case 4.

Please let me know if you have any suggestions for the above two issues. 

Best,
Du

Du,

Converting your dcm to nii with dcm2niix, I agree that the L/R A/P S/I labels are correct. The image origin (0,0,0 point), however, is ~5 slices down from the top of the image. Most brain imaging templates, like we're using internally, expect the image origin to be near the center of the brain. 

The image FOV is much larger than what our program was designed for, and cropping the .nii, as you've suggested, is a good idea. However, I can't confirm whether or not your cropping program has distorted the .nii header geometry, based on this original DICOM. If you want to upload a copy of your cropped .nii, I can take a look at that. You would want to crop the .nii as closely around the head as you can, and also ensure that your program has maintained the correct .nii sform and qform in the new cropped .nii  (so L/R A/P S/I are correct), and the origin ((0,0,0) coordinate) of the new .nii is also vaguely near the center of the brain. When the image origin is outside the image or near the edge of the FOV, image registration is unlikely to work properly because rotations are around a suboptimal axis. We use all internal tools for this sort of thing, so I don't have software to recommend, but I can look at your cropped image and see if it looks correct, if you want to send it to me.

Chris Schwarz

Originally posted by Du Wang:

Thank you, Chris, for explaining those details.

In my initial run, I used raw DICOM data, and most of the cases failed. Specifically, in example case 4, the face and ear mask showed very high intensity, and in case 5, the template face was applied to the wrong area. I've since converted the DICOM to NIfTI and cropped the image to focus only on the head.

I've uploaded the original DICOM for one sample case where the defacing result has a similar issue to example case 4.

Please let me know if you have any suggestions for the above two issues. 

Best,
Du

Hi Chris,

Thank you for clarifying the origin issue. I have attached the cropped NIfTI image for the same case. For your information, to preserve the original properties (size, origin, space) of the image, I simply set all slices below the neck to the background.

Regarding your earlier comment on the mask, "2. Two masks are defined in the template space. One prioritizes the face region in the registration cost function and includes the whole head." Could you please clarify if the term "mask" refers to a de-identification mask? My understanding is that after the template is warped to the input image, a de-identification mask will specify the areas to be replaced, including the face, ears, and area behind the head. Does this mean that MRI_face will generate two distinct de-identification masks?

Best,
Du

Du,
The registration would work more reliably if you would crop the FOV rather than only setting the undesired parts to 0, and also set the origin to roughly the center of the cropped volume (i.e. inside the brain, in roughly the center of the brain). mri_reface has code to detect when the image origin is outside the image or near the edge (I did not confirm whether your image is close enough to the edge to activate this threshold), and it attempts to correct this by setting the origin to the center of the image volume. For your attached .nii, that code would relocate the origin from the current position near the top of the skull to the near the bottom of the neck, neither of which is desirable. Cropping and re-setting the origin would help. Setting the rest to zero, as you did, can help with the cost function once the registration gets going, but it doesn't help with the issue of bad initialization, and rotating around a poor choice of axis, due to coordinate origins that don't line up.

There are not two de-identification masks, but two affine registrations between the input and the template. These masks only influence the weighting of different regions in the affine registration cost function. One emphasizes the face specifically, and one emphasizes the whole tissue volume. Once an affine registration matrix is chosen, the remaining steps are unaffected by which was used to arrive there. 

Best,
Chris Schwarz

Thank you for your suggestion! I will adjust the cropping and settings of my HN dataset accordingly.

Regarding the two affine registrations between the input and the template, are you referring to steps "B) Template co-registered (affine) to input image" and "C) Template warped (non-linear) to input image (only affine transformation in face/ear regions)" described in your manuscript? When performing affine registrations, does mri-reface segment the whole tissue area into different regions to calculate the cost function, with one affine registration specifically emphasizing the face area?

Best,
Du

Two affine registrations are calculated, and the "best" one is chosen internally according to the cost function. From there on, only that one affine is used for all places that affine is used. 

Chris

Thank you for your clarification! 

You mentioned, "These masks only influence the weighting of different regions in the affine registration cost function." Could you please provide more details (in concept, not the algorithm) on how these masks are generated? 

Best,
Du

They're just masks defined once in the MCALT_FaceTemplate space, from a combination of segmentation and hand-drawing. It's nothing fancy or interesting. You can extract them from the binary (rename it to .zip and extract) and look at the .nii files, if you want to.

Thank you so much for taking the time to answer my questions! I appreciate your insights and guidance.

Best,

Du

You're welcome! Please let me know how it performs for you on the cropped-FOV origin-centered images. I think that will be enough to get fairly decently robust performance for you, but our testing with CT has been very limited, so performance reports are helpful.

Chris

No Problem. I'll keep you updated when I get the results.

Best,
Du

Hi Chris, 

I cropped the FOV and set the origin to the center of the cropped volume, but this did not help significantly (only 18 cases were fixed, the remaining 90 more still unresolved). I have attached one of the cropped images for your reference.

Additionally, I noticed that after defacing, the voxel values were altered even in areas outside the deface mask, such as the brainstem. Did the algorithm use any normalization or noise filtering that could impact the voxel values across the entire image?

Best,
Du

Du,
I am happy to take a look at your example, but I don't see any attachment. Can you try again, or email it to me directly? 

Voxel values outside the face and ear regions should not ever be altered. That would only occur if there is a misregistration of some kind. There is not any kind of normalization or noise filtering applied to the rest of the image. 

Best,
Chris