Rails 4 rendering partials with assets extremely slow

Rails 4 rendering partials with assets extremely slow

I recently upgraded our application to Rails 4. Since upgrading rendering
times for a view with image assets loads extremely slowly.
A page with 10 assets can take as much as 8 seconds and some pages oh
heroku will simply timeout due to the rendering taking too long.
I know in Rails 3.2.13 there was an issue with config.assets.debug set to
true by default but that was fixed in Rails 4 I believe.
What could be causing such long rendering times?
Below is the production environment file for reference. Any other details
required, please let me know.
Adventistmedia::Application.configure do
config.cache_classes = true
config.eager_load = true
config.consider_all_requests_local = false
config.action_controller.perform_caching = true
config.cache_store = :dalli_store, {compress: true}
config.serve_static_assets = true # fixes heroku issue
config.static_cache_control = "public, max-age=30758400"
config.assets.js_compressor = :uglifier
config.assets.compile = false
config.assets.digest = true
config.assets.cache_store = :dalli_store
config.assets.debug = false
config.after_initialize do
Delayed::Job.scaler = :heroku_cedar
end
config.assets.precompile += %w(*.png *.jpg *.jpeg *.gif
fontawesome-webfont.ttf fontawesome-webfont.eot FontAwesome.otf
fontawesome-webfont.woff)
config.i18n.fallbacks = true
config.active_support.deprecation = :notify
config.log_formatter = ::Logger::Formatter.new
config.action_controller.asset_host =
"//#{ENV['FOG_DIRECTORY']}.s3.amazonaws.com"
end