I've recently started learning OpenCL for my work at DESY CFEL and found that setting up a C++ environment in XCode for OpenCL is not an entirely straightforward task. Here are some instructions to get started. I am using OSX 10.13.4 (High Sierra) and XCode 9.3.1.
The OpenCL framework in OSX is stored at /System/Library/Frameworks/OpenCL.framework/. Unfortunately, it does not come with a header file for C++ bindings; if you want to use C alone this should not be a problem. I would like to use C++, however.
Download the appropriate cl.hpp file from the Khronos OpenCL Registry. Simply add and include this header file in XCode OpenCL projects and it should work fine. Do not forget to actually link the OpenCL framework under the project settings in XCode. Download and compile this example code, mostly copied from this tutorial, to test your system.
It is possible to add cl.hpp to the system framework so that you do not have to manually add it every time you create a new project. There is one complication: Since OSX 11.11 (El Capitan), Apple has started using something called System Integrity Protection (SIP) to prevent the naive or overly-adventurous from doing too much damage. When activated, it will prevent adding the header to the OpenCL framework even with root access.
This page provides the solution. Restart your system and hold cmd+R to launch OSX in Recovery Mode. Open a Terminal and enter csrutil disable to disable SIP. Restart normally. Move your cl.hpp file to /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/OpenCL.framework/Headers. Return to Recovery Mode and enter csrutil enable in a Terminal if you wish to reenable SIP and restart again. You should now be able to include <OpenCL/cl.hpp> in all of your XCode OpenCL projects.
Notice here that, despite the fact that OpenCL has been released up to version 2.2 as of 2018, Apple devices come with drivers only up to version 1.2. Additionally, it is impossible (as far as I know) for users to update these drivers in OSX. Clearly, Apple has no interest in providing sufficient support for OpenCL. This is due to Apple's development of its Metal framework which, in their eyes, is the obvious choice for anyone developing for the GPU in OSX. Personally, I have no interest in learning an API that is only useful for one system so I hope this is rectified in the future.
References:
Khronos Group
Apple Developer OpenCL Programming Guide for Mac
Using PowderCell to Calculate Angles for Debye-Scherrer Cones
PowderCell is a piece of software that allows calculation of the polar angles of the Debye-Scherrer cones in x-ray powder diffraction. We used these angles to find the alignment of the photon detectors in the experiment I mentioned in a previous post. We used the predicted powder scattering angles from PowderCell, projected them onto a simulated flat detector, and moved the detector in the program until the experimental powder rings lined up with the predicted powder rings. Here is a short tutorial for how to calculate these angles in PowderCell.
You can install Wine on OSX through a number of different methods; I have had success with installing the downloadable binaries and installing through MacPorts and Homebrew. Currently, I am using the MacPorts version.
The first time you start up Wine, it will create a hidden directory, ~/.wine, in ~/ where it will store all of Windows files and programs you will run. The directory structure roughly mimics Windows with directories called drive_c, Program Files, Application Data, and all the other classics more or less in the place you expect. If you ever mess with the directory structure and break things just delete the entire ~/.wine directory and reinstall all of your programs, it will not break anything related to OSX. Installation of programs works just like it would in Windows.
Run Windows programs with Wine by running
Some other important commands that allow you to change various settings such as which Windows version you are running as
Using Wine to Run Windows Programs
PowderCell is a Windows program, but we can still run it in OSX using a cool piece of software called Wine. The short story of Wine is that it will allow you run any Windows software natively in OSX --- no dual-booting Windows necessary! I used to have a Windows partition on my Mac but now I use Wine instead: It is much more convenient works well for running all sorts of programs including your favorite Windows video games.You can install Wine on OSX through a number of different methods; I have had success with installing the downloadable binaries and installing through MacPorts and Homebrew. Currently, I am using the MacPorts version.
The first time you start up Wine, it will create a hidden directory, ~/.wine, in ~/ where it will store all of Windows files and programs you will run. The directory structure roughly mimics Windows with directories called drive_c, Program Files, Application Data, and all the other classics more or less in the place you expect. If you ever mess with the directory structure and break things just delete the entire ~/.wine directory and reinstall all of your programs, it will not break anything related to OSX. Installation of programs works just like it would in Windows.
Run Windows programs with Wine by running
wine program.exe
Some other important commands that allow you to change various settings such as which Windows version you are running as
winecfg
wine regedit
Using PowderCell 2.4
Download PowderCell and run it using Wine. It will ask to extract all of the necessary files to a location of your choosing. After extraction, move to the new directory and run PCW.exe, again with Wine. Some additional comments about running PowderCell with Wine are here.
Now that we have the program running, let's calculate something. In the experimental setup featured in my previous post, we performed x-ray diffraction on powdered lanthanum hexaboride using the LINAC Coherent Light Source at the Stanford Linear Accelerator Laboratory. First, we must download a unit cell .cif file for \(LaB_6\) using the Crystallography Open Database or any other crystallography database your institution gives access to.
PowderCell is an old program and will, unfortunately, not read the .cif file directly. Simply open the unit cell in a text editor and enter the unit cell data manually as pictured below. Be sure to enter the space group correctly. The space group number for \(LaB_6\) is 221. A visualization of your unit cell should automatically be generated after pressing "OK".
In our experiment last month, we used x-rays with energy of 9.43 keV which corresponds to a wavelength of 1.315 angstroms. Under Diffraction >> Experiment, make sure the source is "X-ray" and enter the wavelength in angstroms for your source in the field \(K\alpha_1\). In this window you can also change the range of viewing angles. The peaks of the Debye-Scherrer cones will automatically be plotted for your structure.
PowderCell is an old program and will, unfortunately, not read the .cif file directly. Simply open the unit cell in a text editor and enter the unit cell data manually as pictured below. Be sure to enter the space group correctly. The space group number for \(LaB_6\) is 221. A visualization of your unit cell should automatically be generated after pressing "OK".
The option to view the numerical angles, \(2\theta\), printed above the peaks is given in the right-hand column.
Some Useful Terminal Commands in OSX
These are some commands for the Terminal in Mac OSX that I have found useful and were, at the time, difficult to locate on the Internet. Often, instructions for shell commands are specific to the Linux environment but one must use a different command to complete the same function in OSX; hence, some of the commands below are specific to OSX.
I set the native language on my MacBook to German and, as a result, I have noticed that machines in the United States I need to SSH into (SLAC or MIT computing resources, for example) tend to throw a warning at login or, sometimes, with every command I enter. It is quite annoying and the solution is to add the following to .bash_profile in the home directory.
I have not tested this for other language settings in OSX. However, I imagine this would fix warnings resulting from setting any language (other than English) as the default.
# Show hidden files by default in the Finder
defaults write com.apple.finder AppleShowAllFiles YES
# OSX stores a list of items you download from the Internet and some information about them. View that list with this command.
sqlite3 ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV* 'select LSQuarantineDataURLString from LSQuarantineEvent'
# Delete local database of downloads
sqlite3 ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV* 'delete from LSQuarantineEvent'
# Prevent computer sleep for 600 seconds
caffeinate -u -t 600
# Get local network IP address
ipconfig getifaddr en0
# Get global IP address
curl ipecho.net/plain; echo
# Set Desktop background
osascript -e ‘tell application “Finder” to set desktop picture to POSIX file “/path/to/picture.jpg”’
# Set the default save format of TextEdit to .txt instead of .rtf
defaults write com.apple.TextEdit RichText -int 0
# Disable throttling on low-priority tasks, set back to '1' to reenable
sudo sysctl debug.lowpri_throttle_enabled=0
# Download all .pdf files from a website
wget -A pdf -m -p -E -k -K -np http://
# Check hardware configuration like 'lscpu' in Linux
sysctl hw
# Mute the Mac startup sound
osascript -e 'set volume without output muted'
osascript -e 'set volume with output muted'
# Monitor CPU temperatures from Terminal
gem install iStats
istats
# Create a tar file with gzip compression
tar -czf file.tar.gz files
# Extract tar files
tar -xvf file.tar
# Get a random number, useful in scripts to initialize variables
echo $RANDOM
# Convert image files to new formats
sudo port install convert
convert image.jpg image.png
# Convert PDFs to DJVU files and vice versa
sudo port install pdf2djvu
sudo port install djvu2pdf
pdf2djvu -o out.djvu in.pdf
djvu2pdf in.pdfI set the native language on my MacBook to German and, as a result, I have noticed that machines in the United States I need to SSH into (SLAC or MIT computing resources, for example) tend to throw a warning at login or, sometimes, with every command I enter. It is quite annoying and the solution is to add the following to .bash_profile in the home directory.
export LC_ALL=en_US.UTF-8I have not tested this for other language settings in OSX. However, I imagine this would fix warnings resulting from setting any language (other than English) as the default.
Detector Geometry from Powder Rings
The goal of diffractive imaging experiments is to measure a diffraction pattern on a detector with high precision and calculate the structure in the interaction region (the volume where the structure of interest and the beam intersect) that created the pattern. A critical component of scattering experiments is precise knowledge of the spatial location of pixels on the detector with respect to the interaction region; the interaction region is taken to be the origin of the scattering system. Small tilts or translations of the detector can have huge effects on the spatial location and solid-angle of individual pixels, particularly those close to edge of a flat-panel detector. While Bragg peaks and other features may still be resolved, without a precise and accurate knowledge of the geometry of the detector setup it will be impossible to reconstruct the sample structure because the spatial location of the Bragg peaks on the detector will be unknown.
The question is, how does one measure the spatial location of every pixel on a detector given the limitations of mechanical measurement? We cannot just use mechanical measurements since, ideally, we would like to be able to modify our detector setup to meet different experimental needs without spending hours doing gymnastics with calipers. Additionally, we must consider that individual pixels on the detector will usually have spatial dimensions on the order of ~100 microns and we simply cannot reliably measure these features without optical methods.
One solution is to take a sample with a structure that will produce a known or easily calculated diffraction pattern on all the detectors of interest. One can then compare the predicted diffraction pattern with the measured one and either adjust the detectors to produce the predicted pattern (if the error is very large) or apply mathematical corrections to the model that produced the predicted pattern until it mimics the measured pattern (if the error is small): The sum of these corrections will reveal precisely how the detector is misaligned which tells us exactly how it must be oriented in space.
Note: An assumption we have made is that the in-plane geometry of any single detector, i.e. the coordinates of every pixel on the plane parallel to the detector surface, is known precisely and that we need only track the coordinates and angles or orientation of the detector. This is often a valid assumption since high-resolution detectors always come with geometry specifications from the manufacturer. The location of a detector with respect to other detectors, however, is completely subject to the imagination of the experimenters.
An example of such a method is to take a measurable macroscopic object, such as an ion-milled silicon chip, and place it between the interaction region and the detector of interest. Experimental simulation of a radiating point source at the interaction region will illuminate the silicon chip and, where sections of the chip have been cut through with the ion mill, cast a shadow of the chip on the detector. A precise knowledge of the geometry of the macroscopic object and its orientation with respect to the simulated point source allows the calculation of a predicted shadow. Comparison with the measured shadow will allow a calculation of the orientation and location of the detector. Even measuring the separation of just two points on the projected shadow, with knowledge of the real distance between those corresponding points on the object, allows the valuable calculation of the distance of the detector plane from the interaction region from simple trigonometry. Sometimes, knowledge of the detector distance is all that is needed if detector tilts are clearly small. To get a more accurate knowledge of pixel locations, however, a more sophisticated model of the macroscopic object (such as a 3D scan or computed tomography) is required to construct an accurate model of the shadow and restrict all detector degrees of freedom.
Note: Any physics student will recognize that a perfect, radiating point source is a non-physical phenomena. In a real experiment like this one, light is usually emitted from the entire interaction region. This particular experiment used fluorescent light from an iron salt dissolved in a jetted spray as our "point source" in calibration runs. An x-ray laser beam passes through the jet and causes the iron in the jet solution to fluoresce by exciting core electrons. This light is emitted from every location in space where the beam touches the jet: Since the beam has transverse dimensions on the order of a couple microns, the interaction region is small but certainly not point-like (the transverse dimensions of the jet are on the order of 10-100 microns, much larger than the x-ray beam).
Another common method for producing a predictable pattern on the detector is to use powder diffraction. A powder of a crystalline substance with known scattering angles will produce rings of diffracted light due the random orientation of the crystal planes in the powder with respect to the incident radiation. These rings, neglecting polarization effects, are circularly-symmetric about the axis of the beam and their location on the Ewald sphere is easily calculated. A simulated image of such powder rings for LaB6 powder on a perfect, planar detector (1024 * 1024 pixels) parallel to the beam-axis is shown below.
The measured powder diffraction rings on the same detector during an experiment will look something like the image below.
Note: We have not included, thus far, discussion of the effects of the polarization of the x-ray laser on the powder diffraction pattern or attempted to model this in our predicted diffraction pattern. The polarization effects cause the visible dimming of the rings in the center of the detector and, since this breaks the rotational symmetry, could provide an additional constraint on the degrees of freedom of the detector orientation. I will explain the dimming due to polarization and provide improvements to the geometry fitting based on this effect in future updates. For now, polarization of the incident beam serves simply as an explanation for why the powder rings are not visible across the center of the detector.
One can use a histogram across the detector to see the rings and adjust geometry parameters to make the simulated ring peaks correspond with the measured peaks in the histogram, as below, where the blue line is the measured signal and the green is the ideal, predicted signal across three different slices of the detector.
Though easily implemented, this is not a foolproof method since we cannot view the plot of every slice across the detector simultaneously and so are unlikely to arrive at the perfect fit. Additionally the background noise and diffuse scattering gradient (likely from the capillary that contained the LaB6 powder, which is made of an amorphous material) is not helpful. The image should be cleaned up using localized thresholding to remove the background noise and remove any diffuse scattering or fluorescence.
Overlaying the images in a GUI using something like PyQtGraph allows rapid, fine adjustment of the geometry parameters to fit the measured powder rings. Fitting by visual inspection alone has proven for my purposes sufficient to optimize the geometry with adjustments on the order of \(10^{-2}\) degrees and tens of microns.
To illustrate, here is an example of a horribly misaligned detector geometry and the discrepancy between the measured and predicted powder rings.
As you might imagine, the actual improvement in goodness of fit is difficult to evaluate for fine parameter adjustments. Below is a rough detector geometry followed by an improved geometry I produced with my PyQtGraph GUI application.
The improvement is barely perceptible but close inspection should show that the bottom image has been shifted very slightly and the overlapping region of the predicted and measured rings is closer to center compared to the previous image.
Perhaps a simple measure of the average brightness of the composite image could be a naive quantification of the goodness of fit (since the composite image brightens where the measured and predicted rings overlap). Likely, though, a more sophisticated method would be necessary to achieve a fit of any respectable precision. For the present, a manual fit by visual inspection seems good enough for our research purposes.
This post was inspired by my recent work at the Center for Free-Electron Laser Science, a section of the Deutsches Elektronen Synchrotron in Hamburg. I worked with Fabian Trost and Kartik Ayyer in analyzing this and other data from our group's experiments at the LINAC Coherent Light Source at the Stanford Linear Accelerator.
The question is, how does one measure the spatial location of every pixel on a detector given the limitations of mechanical measurement? We cannot just use mechanical measurements since, ideally, we would like to be able to modify our detector setup to meet different experimental needs without spending hours doing gymnastics with calipers. Additionally, we must consider that individual pixels on the detector will usually have spatial dimensions on the order of ~100 microns and we simply cannot reliably measure these features without optical methods.
One solution is to take a sample with a structure that will produce a known or easily calculated diffraction pattern on all the detectors of interest. One can then compare the predicted diffraction pattern with the measured one and either adjust the detectors to produce the predicted pattern (if the error is very large) or apply mathematical corrections to the model that produced the predicted pattern until it mimics the measured pattern (if the error is small): The sum of these corrections will reveal precisely how the detector is misaligned which tells us exactly how it must be oriented in space.
Note: An assumption we have made is that the in-plane geometry of any single detector, i.e. the coordinates of every pixel on the plane parallel to the detector surface, is known precisely and that we need only track the coordinates and angles or orientation of the detector. This is often a valid assumption since high-resolution detectors always come with geometry specifications from the manufacturer. The location of a detector with respect to other detectors, however, is completely subject to the imagination of the experimenters.
An example of such a method is to take a measurable macroscopic object, such as an ion-milled silicon chip, and place it between the interaction region and the detector of interest. Experimental simulation of a radiating point source at the interaction region will illuminate the silicon chip and, where sections of the chip have been cut through with the ion mill, cast a shadow of the chip on the detector. A precise knowledge of the geometry of the macroscopic object and its orientation with respect to the simulated point source allows the calculation of a predicted shadow. Comparison with the measured shadow will allow a calculation of the orientation and location of the detector. Even measuring the separation of just two points on the projected shadow, with knowledge of the real distance between those corresponding points on the object, allows the valuable calculation of the distance of the detector plane from the interaction region from simple trigonometry. Sometimes, knowledge of the detector distance is all that is needed if detector tilts are clearly small. To get a more accurate knowledge of pixel locations, however, a more sophisticated model of the macroscopic object (such as a 3D scan or computed tomography) is required to construct an accurate model of the shadow and restrict all detector degrees of freedom.
Note: Any physics student will recognize that a perfect, radiating point source is a non-physical phenomena. In a real experiment like this one, light is usually emitted from the entire interaction region. This particular experiment used fluorescent light from an iron salt dissolved in a jetted spray as our "point source" in calibration runs. An x-ray laser beam passes through the jet and causes the iron in the jet solution to fluoresce by exciting core electrons. This light is emitted from every location in space where the beam touches the jet: Since the beam has transverse dimensions on the order of a couple microns, the interaction region is small but certainly not point-like (the transverse dimensions of the jet are on the order of 10-100 microns, much larger than the x-ray beam).
Another common method for producing a predictable pattern on the detector is to use powder diffraction. A powder of a crystalline substance with known scattering angles will produce rings of diffracted light due the random orientation of the crystal planes in the powder with respect to the incident radiation. These rings, neglecting polarization effects, are circularly-symmetric about the axis of the beam and their location on the Ewald sphere is easily calculated. A simulated image of such powder rings for LaB6 powder on a perfect, planar detector (1024 * 1024 pixels) parallel to the beam-axis is shown below.
The measured powder diffraction rings on the same detector during an experiment will look something like the image below.
Note: We have not included, thus far, discussion of the effects of the polarization of the x-ray laser on the powder diffraction pattern or attempted to model this in our predicted diffraction pattern. The polarization effects cause the visible dimming of the rings in the center of the detector and, since this breaks the rotational symmetry, could provide an additional constraint on the degrees of freedom of the detector orientation. I will explain the dimming due to polarization and provide improvements to the geometry fitting based on this effect in future updates. For now, polarization of the incident beam serves simply as an explanation for why the powder rings are not visible across the center of the detector.
One can use a histogram across the detector to see the rings and adjust geometry parameters to make the simulated ring peaks correspond with the measured peaks in the histogram, as below, where the blue line is the measured signal and the green is the ideal, predicted signal across three different slices of the detector.
Though easily implemented, this is not a foolproof method since we cannot view the plot of every slice across the detector simultaneously and so are unlikely to arrive at the perfect fit. Additionally the background noise and diffuse scattering gradient (likely from the capillary that contained the LaB6 powder, which is made of an amorphous material) is not helpful. The image should be cleaned up using localized thresholding to remove the background noise and remove any diffuse scattering or fluorescence.
Overlaying the images in a GUI using something like PyQtGraph allows rapid, fine adjustment of the geometry parameters to fit the measured powder rings. Fitting by visual inspection alone has proven for my purposes sufficient to optimize the geometry with adjustments on the order of \(10^{-2}\) degrees and tens of microns.
As you might imagine, the actual improvement in goodness of fit is difficult to evaluate for fine parameter adjustments. Below is a rough detector geometry followed by an improved geometry I produced with my PyQtGraph GUI application.
The improvement is barely perceptible but close inspection should show that the bottom image has been shifted very slightly and the overlapping region of the predicted and measured rings is closer to center compared to the previous image.
Perhaps a simple measure of the average brightness of the composite image could be a naive quantification of the goodness of fit (since the composite image brightens where the measured and predicted rings overlap). Likely, though, a more sophisticated method would be necessary to achieve a fit of any respectable precision. For the present, a manual fit by visual inspection seems good enough for our research purposes.
This post was inspired by my recent work at the Center for Free-Electron Laser Science, a section of the Deutsches Elektronen Synchrotron in Hamburg. I worked with Fabian Trost and Kartik Ayyer in analyzing this and other data from our group's experiments at the LINAC Coherent Light Source at the Stanford Linear Accelerator.
Setting Up the Terminal in OSX
I use the Terminal in Mac OSX every day for my work and there are some small hacks that make the experience of using and coding in the Terminal much more bearable. In particular, enabling colors, some key aliases, and syntax highlighting are essentials for me on every Unix/Linux machine I use.
You can copy-paste the code below. (For those who are new to the Terminal, use "ls -a" to show hidden directories and files, i.e. those files and directories with a leading "." character in their name like .bash_profile). The first section enables colors, which should subsequently be modifiable under Terminal > Settings > Profile by messing around with the ANSI Colors section for the appropriate terminal profile. The "Show ANSI Colors" option must also be enabled on this page.
The second section enables aliases: These are shortcuts for Terminal commands (which can often be quite long) that, when typed into the Terminal, execute the more complex command. The concept and name are intuitive. For example, with the aliases above "ls" gets substituted by "ls -HFG" every time I want to use "ls". I find this particularly useful for ssh commands since I need to log into a lot of computational mainframes and remembering all their IP addresses is unreliable.
In the Terminal, install an updated version of nano using Homebrew. This creates /usr/local/share/nano which contains a number of common syntax files. We need to create a .nanorc file in the home directory which configures nano to include all of the available syntax highlights. The StackExchange article provides the relevant command; the code summary is reproduced below.
Colors and Aliases
Colors and aliases require a simple addition to the .bash_profile in your home directory. First, you need to make sure that you have the GNU CoreUtils installed. If you are not yet using Homebrew for Mac OSX, you should be. In the Terminal, run "brew install coreutils" to install GNU CoreUtils. (I may do a post in the future about Homebrew and the Unix/Linux Terminal in the future, but the basics are ubiquitous on the web).You can copy-paste the code below. (For those who are new to the Terminal, use "ls -a" to show hidden directories and files, i.e. those files and directories with a leading "." character in their name like .bash_profile). The first section enables colors, which should subsequently be modifiable under Terminal > Settings > Profile by messing around with the ANSI Colors section for the appropriate terminal profile. The "Show ANSI Colors" option must also be enabled on this page.
# Terminal colors (after installing GNU coreutils)
NM="\[\033[0;38m\]"
HI="\[\033[0;37m\]"
HII="\[\033[0;31m\]"
SI="\[\033[0;33m\]"
IN="\[\033[0m\]"
export PS1="$NM[ $HI\u $HII\h $SI\w$NM ] $IN"
export CLICOLORk=1
export LSCOLORS=ExFxBxDxCxegedabagacad
# Useful aliases
alias ls="ls -HFG"
alias cd..="cd .."
alias c="clear"
alias e="exit"
alias ssh="ssh -XY"
alias ..="cd .."
alias ssh_command="ssh -XY user@login.example.org"The second section enables aliases: These are shortcuts for Terminal commands (which can often be quite long) that, when typed into the Terminal, execute the more complex command. The concept and name are intuitive. For example, with the aliases above "ls" gets substituted by "ls -HFG" every time I want to use "ls". I find this particularly useful for ssh commands since I need to log into a lot of computational mainframes and remembering all their IP addresses is unreliable.
Syntax Highlighting in Nano
I like to use nano as my default text editor in the Terminal: It is simple to use and quite frankly I have not gotten around to memorizing all those emacs and vim shortcuts. One of the drawbacks to nano compared to these other editors is that it has no code syntax highlighting. As this StackExchange article shows, it is possible to add it using Homebrew.In the Terminal, install an updated version of nano using Homebrew. This creates /usr/local/share/nano which contains a number of common syntax files. We need to create a .nanorc file in the home directory which configures nano to include all of the available syntax highlights. The StackExchange article provides the relevant command; the code summary is reproduced below.
brew install nano
/bin/ls /usr/local/share/nano/*.nanorc | xargs -I {} echo 'include "{}"' >> ~/.nanorc
source ~/.nanorc
Subscribe to:
Posts (Atom)