A piggy bank of commands, fixes, succinct reviews, some mini articles and technical opinions from a (mostly) Perl developer.

Jump to

Quick reference

Showing posts with label file. Show all posts
Showing posts with label file. Show all posts

How to really loop over filenames with spaces in

 Don't parse the output of "ls".

Instead:

for file in *

do

  ...

done


(source)

Linux renaming examples, when filenames have spaces

Do your filenames have spaces? Do you curse bash for making it difficult to loop over them?

Fear not! (and wash that potty mouth out with soap). This loop is all you need:

find . -type f -name "*.mp3" -print0 | while IFS= read -r -d '' file; do ls "$file"; done

# Rename "foo.mp3.mp3" to "foo.mp3"

find . -type f -name "*.mp3" -print0 | while IFS= read -r -d '' file; do file=`basename "$file"`; newfile=$(perl -le'$ARGV[0]=~s/\.mp3//;print$ARGV[0]' "$file"); echo mv "$file" "$newfile"; done


# Add a leading zero to numbered filenames

find . -type f -name "064*" -print0 | while IFS= read -r -d '' file; do file=`basename "$file"`; echo mv "$file" "0$file"; done

# The above commands will print out what they're about to do. Check it is what you expect! Then make this change to run them for real:

Change: echo mv foo bar
To: `mv foo bar`

# ^ Those ` are backticks

# Note the order of files is not sorted


Flat file vs database

Flat file
  • Easy to set up, only have to consider local file permissions
  • Easy to implement in an ad-hoc way, ideal for a prototype
    • Plain text for very simple things like a list
    • JSON/YAML/Perl for complex data structures
  • Doesn't work when the app is load balanced across multiple servers
  • Not automatically backed up
  • Amazon S3 is relatively expensive
  • Have to make your own model
Database
  • Requires up-front schema design - more work, but forces you to consider design of data 
  • You can put business logic in the ResultSet models
  • Requires an instance to be provisioned
  • Easy cross-referencing of data
  • Works when the app is load balanced across multiple servers
  • Backup-as-a-service (i.e. replication)
  • Amazon RDS is cheaper than S3
  • Get the model for free with ORM
Conclusion

For production services that have redundancy (load balanced), always use a database unless the overhead of setting one up for the first time is considered too high for the business.

Windows: folder or file open in another program

When you try to move a folder:

    "The action can't be completed because the folder or a file in it is open in another program"

This stunningly unhelpful error message has persisted through many versions of Windows.

The way I work around it is:

  • Rename an unrelated file or folder nearby, then rename it back again.
  • The original folder or file should now be "unlocked"
You can always reboot to fix it, too.

Different files for log4perl levels

log4perl.rootLogger=DEBUG,CONSOLE,ViewableLog,DebugLog,ErrorLog

###################################################################################
# console - what to watch
log4perl.appender.CONSOLE=Log::Log4perl::Appender::Screen
log4perl.appender.CONSOLE.layout=PatternLayout
log4perl.appender.CONSOLE.layout.ConversionPattern=[%d] [%p %c] - %m%n
log4perl.appender.CONSOLE.Filter = ViewableFilter

  log4perl.filter.ViewableFilter        = sub {    \
       my %p = @_;                           \
       $p{log4p_level} eq "FATAL" or          \
       $p{log4p_level} eq "ERROR" or          \
       $p{log4p_level} eq "INFO" or          \
       $p{log4p_level} eq "WARN"           \
                                          }

###################################################################################
# Viewable log - record of CONSOLE
log4perl.appender.ViewableLog=Log::Log4perl::Appender::File
log4perl.appender.ViewableLog.layout=PatternLayout
log4perl.appender.ViewableLog.layout.ConversionPattern=[%d] [%p %c] - %m%n
log4perl.appender.ViewableLog.filename = logs/output.log
log4perl.appender.ViewableLog.Filter = ViewableFilter

###################################################################################
# Error log - warnings and errors
log4perl.appender.ErrorLog=Log::Log4perl::Appender::File
log4perl.appender.ErrorLog.layout=PatternLayout
log4perl.appender.ErrorLog.layout.ConversionPattern=[%d] [%p %c] - %m%n
log4perl.appender.ErrorLog.filename = logs/error.log
log4perl.appender.ErrorLog.Filter = ErrorFilter

  log4perl.filter.ErrorFilter        = sub {    \
       my %p = @_;                           \
       $p{log4p_level} eq "FATAL" or          \
       $p{log4p_level} eq "ERROR" or          \
       $p{log4p_level} eq "WARN"           \
                                          }

###################################################################################
# Debug Log - everything
 log4perl.appender.DebugLog = Log::Log4perl::Appender::File
 log4perl.appender.DebugLog.layout=PatternLayout
 log4perl.appender.DebugLog.layout.ConversionPattern=[%d] [%p %c] - %m%n
  log4perl.appender.DebugLog.filename = logs/debug.log
  log4perl.appender.DebugLog.Filter = AllFilter

  log4perl.filter.AllFilter        = sub {    \
       my %p = @_;                           \
       $p{log4p_level} eq "FATAL" or          \
       $p{log4p_level} eq "ERROR" or          \
       $p{log4p_level} eq "WARN" or          \
       $p{log4p_level} eq "INFO" or          \
       $p{log4p_level} eq "DEBUG" or          \
       $p{log4p_level} eq "TRACE"           \
                                          }

###################################################################################
# To enable DEBUG-level logging for a particular module (and any modules that subclass it),
# un-comment the following and change e.g. 'TempTopicsAdminTools.ModuleManager' to the your own module.

##log4perl.logger.TempTopicsAdminTools.ModuleManager=DEBUG
#log4perl.logger.TempTopicsAdminTools.PageManager=WARN
#log4perl.logger.TempTopicsAdminTools.CategoryManager=WARN