#+Title: Shell scripts in Haskell & turtle?
#+Author: Yann Esposito
#+Date: <2018-11-07 Wed>

*  Script shell en Haskell: Pourquoi?

*** Description

#+BEGIN_QUOTE
turtle is a reimplementation of the Unix command line environment in Haskell so
that you can use Haskell as both a shell and a scripting language.

Features include:

  * Batteries included: Command an extended suite of predefined utilities
  * Interoperability: You can still run external shell commands
  * Portability: Works on Windows, OS X, and Linux
  * Exception safety: Safely acquire and release resources
  * Streaming: Transform or fold command output in constant space
  * Patterns: Use typed regular expressions that can parse structured values
  * Formatting: Type-safe printf-style text formatting
  * Modern: Supports text and system-filepath
#+END_QUOTE

*** Pourquoi?

- Facile à maintenir, à refactorer (fp + types)
- programme type checké et interprété rapidement (<1s)
- syntaxe "légère", distinction string, variables, etc..
- scripts shell sont seulement interprétés et lents;
  scripts Haskell peuvent être interpretés & lent ou compilés & rapide

Parfait pour remplacer de *gros* (> 100 lignes) et *lents* scripts bash.

*** Shell: /Good/

1. commandes internes
2. commandes externes
3. composition:
   - redirection de la sortie standard
   - pipe
   - async

*** Shell: *BAD*

1. variables (syntaxe $FOO, $foo, etc...)
2. gestion des string (for i in *; do ...; done) (files with ' ' or "\n",
   etc...)
3. modularité du code, fonctions
4. Vitesse d'exécution

** Haskell Turtle

*** Commandes Internes

=Turtle.Prelude=

- =cd=, =ls=, =find=, =pwd=, =chmod=, etc...
- =sleep=, =echo=, =err=, =arguments=, =realpath=
- =mv=, =mkdir=, =mktree= (=mkdir -p=)
- =cp=, =cptree=, =symlink=
- =date=, =datefile=
- =which=, =whichAll= (tous les PATHs)

*** commandes externes

- beaucoups de variantes plus ou moins "safes", "génériques"

#+BEGIN_SRC haskell
inshell :: Text -- ^ Command Line
        -> Shell Line -- ^ Lines of stdin
        -> Shell Line -- ^ Lines of stdout
#+END_SRC

- variantes avec separation des arguments =shell= vs =proc=
- variantes strictes (stream)
- avec sans, stderr
- avec sans gestion des valeurs de retour

*** commandes externes (2)

La forme la plus générique:

#+BEGIN_SRC haskell
procStrictWithErr :: MonadIO io
  => Text       -- ^ Command
  -> [Text]     -- ^ Arguments
  -> Shell Line -- ^ Lines of standard input
  -> io (ExitCode, Text, Text) -- ^ (Exit code, stdout, stderr)
#+END_SRC

Run a command using execvp, retrieving the exit code, stdout, and stderr as a
non-lazy blob of Text

*** composition: redirections

#+BEGIN_SRC bash
cmd < input.txt > output.txt
#+END_SRC

Notion de stream de valeurs, generalement: =Shell Line=

#+BEGIN_SRC haskell
-- wc -l <input.txt >output.txt
inshell "wc -l" (input "input.txt") (output "output.txt")
#+END_SRC

*** composition: pipes

Bash:
#+BEGIN_SRC bash
ls | grep foo
#+END_SRC

Turtle:
#+BEGIN_SRC haskell
ls & grep "foo"
#+END_SRC

*** composition: async

Bash:
#+BEGIN_SRC bash
sleep 1; echo "foo" &
#+END_SRC

Turtle:
#+BEGIN_SRC haskell
fork (do {shell "sleep 1" empty; echo "foo"})
#+END_SRC

*** composition: async

#+BEGIN_SRC haskell
>>> :set +m
>>> runManaged $ do
  fork $ do
     echo "start"
     sleep 1
     echo "foo"
     sleep 2
     echo "done"
  echo "waiting a bit"
  sleep 2
  echo "stop waiting"
waitsitnagr ta
 bit
foo
stop waiting
#+END_SRC

*** Securité des resources =Managed=

- take a resource and think to free it:
  - =readonly=
  - =writeonly=
  - =appendonly=
- =mktemp= create a temporary file, handle race condition
- =fork=, =wait=
- =pushd=; =view (pushd "/tmp" >> "pwd")= (=> =FilePath "/tmp"=) =pwd= (=FilePath "/"=)

*** Fold

=Foldl= pour faire des accumulations efficacement (plus vite qu'en C) et en un
seul parcours.

#+BEGIN_SRC haskell
>>> fold (ls "/tmp") ((,,) <$> Fold.length <*> Fold.head <*> Fold.last)
(21,Just (FilePath "/tmp/aaa"),Just (FilePath "/tmp/zzz"))
#+END_SRC

** Gestion des arguments

*** Example (1/2)

=options :: MonadIO io => Description -> Parser a -> io a=

#+BEGIN_SRC haskell
parser :: Parser (Text, Int)
parser = (,) <$> optText "name" 'n' "Your first name"
             <*> optInt  "age"  'a' "Your current age"

main = do
    (name, age) <- options "Greeting script" parser
    echo (repr (format ("Hello there, "%s) name))
    echo (repr (format ("You are "%d%" years old") age))
#+END_SRC

*** Example (2/2)

#+BEGIN_SRC
> ./options --name John --age 42
Hello there, John
You are 42 years old
#+END_SRC

#+BEGIN_SRC
> ./options --help
Greeting script

Usage: options (-n|--name NAME) (-a|--age AGE)

Available options:
 -h,--help                Show this help text
 --name NAME              Your first name
 --age AGE                Your current age
#+END_SRC

*** Real World™ Complex Example (1/4)

#+BEGIN_SRC haskell
gpm :: IO ()
gpm = do
  subcmd <- options "Git Project Manager" parser
  case subcmd of
   Init              -> Init.init
   NewIssue issueOpt -> inGPM (Issue.handleNewIssue issueOpt)
   Review reviewCmd  -> inGPM (Review.handleReview reviewCmd)
   Serve serveCmd    -> Serve.handleServe serveCmd
   Hooks hooksCmd    -> inGPM (Hooks.handleHooks hooksCmd)

data Command = Init
             | NewIssue Issue.IssueOptions
             | Review Review.ReviewCommand
             | Serve Serve.ServeCommand
             | Hooks Hooks.HooksCommand

parser :: Parser Command
parser = subcommand "init" "Initialize gpm" (pure Init)
         <|> NewIssue <$> subcommand "new-issue"
                                     "Create a new Issue"
                                     Issue.parseIssueOptions
         <|> Review <$> subcommand "review"
                         "Review (use current branch by default)"
                         Review.parseReviewCmd
         <|> Serve <$> subcommand "serve"
                         "Serve the git to the web"
                         Serve.parseServeCommand
         <|> Hooks <$> subcommand "hooks"
                         "Handle hooks for this git repository"
                         Hooks.parseHooksCommand
#+END_SRC

*** Real World™ Complex Example (2/4)

#+BEGIN_SRC bash
> gpm
Git Project Manager

Usage: gpm (init | new-issue | review | serve | hooks)

Available options:
  -h,--help                Show this help text

Available commands:
  init                     Initialize gpm
  new-issue                Create a new Issue
  review                   Review (use current branch by default)
  serve                    Serve the git to the web
  hooks                    Handle hooks for this git repository
#+END_SRC

*** Real World™ Complex Example (3/4)

#+BEGIN_SRC haskell
data IssueOptions = IssueOptions
                    { interactive :: Bool
                    , newIssue    :: NewIssue
                    }


parseIssueOptions :: Parser IssueOptions
parseIssueOptions = IssueOptions
                    <$> switch "interactive" 'i' "Interactive mode"
                    <*> parseNewIssue

parseNewIssue :: Parser NewIssue
parseNewIssue = do
  isPriority    <- optional $ optText "priority"  'p' "Priority A,B,C"
  isStatus      <- optional $ optText "status"    's' "The status of the issue (TODO, QUESTION, ...)"
  isTitle       <- optional $ optText "title"     't' "The status title"
  isUser        <- optional $ optText "creator"   'c' "The user that created the issue"
  isBranch      <- optional $ optText "branch"    'b' "The branch related to the issue"
  isTags        <- optional $ optText "tags"      'g' "comma separated tags"
  isAssignee    <- optional $ optText "assignee"  'a' "Assignee"
  isReviewers   <- optional $ optText "reviewers" 'r' "comma separated reviewers"
  isDescription <- optional $ optText "descr"     'd' "Long issue description"
  pure NewIssue { priority = maybe PriorityB toPriority isPriority
                , status = fromMaybe "TODO" isStatus
                , title  = fromMaybe "Issue Title" isTitle
                , user        = isUser
                , branch      = isBranch
                , tags        = maybe [] (T.splitOn ",") isTags
                , assignee    = isAssignee
                , reviewers   = maybe [] (T.splitOn ",") isReviewers
                , description = isDescription
                }
#+END_SRC

*** Real World™ Complex Example (4/4)

#+BEGIN_SRC text
> gpm new-issue --help
Usage: gpm new-issue [-i|--interactive] [-p|--priority PRIORITY]
                     [-s|--status STATUS] [-t|--title TITLE]
                     [-c|--creator CREATOR] [-b|--branch BRANCH]
                     [-g|--tags TAGS] [-a|--assignee ASSIGNEE]
                     [-r|--reviewers REVIEWERS] [-d|--descr DESCR]
  Create a new Issue

Available options:
  -i,--interactive         Interactive mode
  -p,--priority PRIORITY   Priority A,B,C
  -s,--status STATUS       The status of the issue (TODO, QUESTION, ...)
  -t,--title TITLE         The status title
  -c,--creator CREATOR     The user that created the issue
  -b,--branch BRANCH       The branch related to the issue
  -g,--tags TAGS           comma separated tags
  -a,--assignee ASSIGNEE   Assignee
  -r,--reviewers REVIEWERS comma separated reviewers
  -d,--descr DESCR         Long issue description
  -h,--help                Show this help text
#+END_SRC

** Regex? NO; Parsers!

*** match

#+BEGIN_SRC haskell
>>> match ("can" <|> "cat") "cat"
["cat"]
#+END_SRC

Parser:

#+BEGIN_SRC haskell
("can" <|> "cat")
#+END_SRC

*** Parser composition

=Pattern a=: Polymorphique =a=

- Functor
- Applicative (=*>=)
- Monad
- Alternative (=<|>=)
- MonadPlus

- Monoid a => Monoid (Pattern a)
- Monoid a => SemiGroup (Pattern a)

- Monoid a => Num (Pattern a); =+= (<=> =<|>=), =*= (<=> =<>=)

*** sed: Pattern Text

#+BEGIN_SRC haskell
sed :: Pattern Text -> Shell Line -> Shell Line
#+END_SRC

#+BEGIN_SRC haskell
>>> stdout (input "file.txt")
Test
ABC
42
>>> -- sed 's/C/D/g' file.txt
>>> stdout (sed ("C" *> return "D") (input "file.txt"))
Test
ABD
42
#+END_SRC

*** sed: Apply fn on matches

#+BEGIN_SRC haskell
>>> -- sed 's/[[:digit:]]/!/g' file.txt
>>> stdout (sed (digit *> return "!") (input "file.txt"))
Test
ABC
!!
>>> import qualified Data.Text as Text
>>> -- rev file.txt
>>> stdout (sed (fmap Text.reverse (plus dot)) (input "file.txt"))
tseT
CBA
24
#+END_SRC

*** Helpers

- =inplace=: =perl -pi -e '...' file=
- =find :: Pattern a -> FilePath -> Shell FilePath=
- =findtree :: Pattern a -> Shell FilePath -> Shell FilePath= (ie =ls | grep=)

** Conclusion

/SAFE/ - /FAST/ - /AWESOME/ -> /CONFIDENCE/

  * /Batteries included/: Command an extended suite of predefined utilities
  * /Interoperability/: You can still run external shell commands
  * /Portability/: Works on Windows, OS X, and Linux
  * /Exception safety/: Safely acquire and release resources
  * /Streaming/: Transform or fold command output in constant space
  * /Patterns/: Use typed regular expressions that can parse structured values
  * /Formatting/: Type-safe printf-style text formatting
  * /Modern/: Supports text and system-filepath